Mustafa Demir
Department of Mathematics, Computer Science & Software Engineering
University of Detroit Mercy
United States demirmf@udmercy.edu
Abstract : This study examined the effects of using Virtual Manipulatives (VMs) on students’ solutions of the mathematical problems requiring different levels of cognitive effort. Through using the categorization of tasks developed by Smith and Stein (1998), the problems are identified as procedures without connections, procedures with connections, and doing mathematics. Participants were forty-eight college students taking a remedial mathematics course, randomly assigned into two groups, and used the same set of VMs with open-ended exploratory versus structured mathematics questions. The pre-and posttest assessing students’ performance on the three types of questions mentioned above were administered and students’ interactions with the VMs were analyzed. The findings revealed that students using VMs with the open-ended activities considerably improved their solutions on the “doing mathematics” questions requiring them to understand the nature of mathematical ideas and processes.
A variety of information technology has been mostly used to support mathematics instruction in the last two decades. With the latest developments in computer and Internet technologies, manipulations of objects on the computer screen have become easier and faster than those on physical manipulatives. These web-based interactive objects called Virtual Manipulatives (VMs) provide multiple-linked representations of mathematical concepts. While working them, learners can make changes on a particular representation and instantly observe the effects of their changes on the other representations. Therefore, using VMs enables learners to develop and test their hypotheses through exploring the relationships among various representations of mathematics concepts. NCTM’s Technology Principle also emphasizes that “Work with virtual manipulatives…. can allow young children to extend physical experience and to develop an initial understanding of sophisticated ideas like the use of algorithms” (NCTM, 2000, p. 26-27). While exploring mathematics concepts through direct manipulation on VMs, children can learn concepts that were previously thought as ‘too advanced’ for them
(Resnick et al., 1998). Thus, VMs can be considered as cognitive tools (Zbiek et al., 2007) to enhance mathematics learning and teaching.
As cognitive tools, VMs can also reduce learners’ cognitive load by providing an interactive environment where learners can directly work with multiple-linked on-screen representations of math concepts.
Multiple-linked representations can allow learners to overcome their difficulty of having further cognitive load in their learning process. Through working with multiple-linked representations, learners can perform action(s) on one representation and instantly observe the effects of their action(s) on another representation. As the translations between representations were automatically performed by the multiple-linked representations system, it can be assumed that cognitive load placed on learners should be decreased and thus learners can easily concentrate on the relations among representations (Kaput, 1992; Scaife & Roger, 1996). Although multiple-linked representations can decrease the cognitive load while studying mathematical ideas, learners often cannot automatically make the connection between their actions with manipulatives and symbols (Kaput,
1989). Thus, it is essential to design instructional activities that will enable students to identify the relations between their actions with the VMs and mathematical ideas.
Although VMs provide unique affordances to enhance students’ mathematics learning, there is a substantial need to design effective instructional activities that will enable students to maximize their learning from VMs. After reviewing the research studies on learning and technology, Clark (1983) concluded that it is not the technology per se that influences learning, it is the instruction used with technology affects learning.

Most tasks used in the technological settings aim to help students build their own experiences, thus computerbased instructional activities should be designed based on learners’ experiences and the properties of the tools of the technology they are using (Sinclair, 2006).
Numerous researchers (e.g., already examined the impact of VMs on students learning of varied mathematics concepts; however, these studies almost paid no attention to design of instructional activities or even modify traditional instructional tasks based on the unique properties of VMs to improve students’ effective use of VMs in their learning. For example, Suh and Moyer (2007) analyzed students’ learning of algebraic relationships such as forming and representing linear equations while working with VMs and physical manipulatives. In the study, students interacted with VMs throughout the same instructional activities that were used with physical manipulatives. In particular, one group of students used virtual balance applet to form the algebraic equations (e.g., x + 2 = 9) and find the missing numbers in these equations. Similarly, students using the physical manipulatives tried to solve the same type of questions as the students working with the VMs.
VMs are often designed to support student’ understanding of difficult mathematics concepts. Slope is one such concept that provides an important foundation for the learning of advanced mathematics concepts
(Thompson, 1994). Research has pointed out students’ various difficulties in learning slope. The most common ones are slope versus height confusion (McDermott et al., 1987); making connections from an algebraic formula of slope to its graphical (Leinhardt et al., 1990) and functional (Lobato et al., 2003) representations; and interpreting slope as a measure of rate of change (Stump, 2001).
Although numerous research (e.g., Crawford & Brown, 2003; Suh et al., 2005; Suh & Moyer, 2007;
Moyer et al., 2008) often underlined the considerable potential of VMs to enhance students’ higher-level cognitive skills, they rarely examined the impact of VMs on students’ performances throughout the mathematical problems requiring various levels of cognitive demands. Therefore, this study examined the impact of using VMs with two different instructional approaches on students’ solutions of the problems with different levels of cognitive demand.
This study aimed to increase our understanding of using VMs for helping students overcome their difficulties in solving questions that require different levels of cognitive demands. Smith & Stein (1998) classified mathematical problems based on their level of cognitive demand. Having applied their categorization to the questions used in this study, three types of problems were identified. These are Procedures without
Connections (PWOC) , Procedures with Connections (PC), and Doing Mathematics (DM) . Smith & Stein considered the PWOC questions as the questions that require finding only correct answers with no or limited explanation. Thus, PWOC items were viewed as the tasks that involve lower-level demands. On the other hand, they classified PC and DM type of questions as the tasks requiring higher-level demands. In particular, PC items require students to work with the conceptual ideas behind the procedures, thus these items aim to develop students’ comprehension. Similarly, while solving DM type of questions, students are expected to analyze and identify the nature of mathematical concepts, processes and relationships.
In the study, students worked with a set of VMs based on two different instructional approaches to improve their learning of slope. In one approach, students used VMs to answer the open-ended exploratory questions asking them to observe the activities on the VMs, make reflection on their interactions with the VMs, and identify the relations among various mathematical ideas based on their observations. In the other instructional approach, students worked with the same set of VMs that OVM group used to respond the structured mathematics questions requiring them to mostly focus on numeric expressions, quantitative relations, calculations and mathematical procedures.
These two instructional approaches have been mostly used in mathematics instruction and they can be thought as examples of two main competing approaches in mathematics education, often called traditional versus reform-based math. Thus, this study will reveal the effects of using the two different instructions that exemplify two leading and contrasting perspectives of mathematics teaching with the same set of VMs on students’ cognitive abilities while solving the questions measuring their learning of slope.
In particular, the study aimed to identify the impact of using VMs with open-ended exploratory versus structured mathematics tasks on students’ overall pre- to posttest gain scores and their gains in three types of questions; procedures with connections, procedures without connections, and doing mathematics.

Forty-eight students taking a remedial mathematics courses (Intermediate Algebra) at a large university in the Midwestern United States completed all sessions of the study. Each student participated in six sessions, one to complete a background questionnaire and paper-pencil pretest on slope, four for the intervention, and one to complete the posttest (see Table 1). After completing the first session, students were randomly assigned into two groups: VMs with open-ended exploratory questions (OVM), and VMs with structured mathematics questions (SVM).
Sessions Groups
VMs with VMs with
Pretest
Open-Ended Q.
OVM
Structured Q.
SVM
Int. S. 1
Int. S. 2
Int. S. 3
Int. S. 4
OVM
OVM
OVM
OVM
SVM
SVM
SVM
SVM
Posttest OVM SVM
Table 1 : Design of the study for the VMs with open-ended versus structured mathematics questions.
In the four 30- to 45-minute intervention sessions, OVM students used a set of VMs to answer openended exploratory questions asking them to observe the activities on the VMs, reflect their interactions with
VMs, and identify the relations among different mathematical ideas based on their observations. SVM students worked with the same set of VMs to respond the structured mathematics questions that entailed using formal mathematics language such as numeric values, calculations, and mathematical procedures. The questions for all students were presented on the right side of the computer screen, next to the VMs, with students typing their responses on the computer (see Figure 1).
Figure 1 : A screenshot of the VM with open-ended exploratory questions.
The students’ responses on the pretest, four intervention sessions, and posttest were analyzed. The preand posttest consisted of questions with three different levels of cognitive demand; procedures without connections, procedures with connections, and doing mathematics.

A scoring guideline was developed to evaluate students’ answers to the pre-and posttest items. To determine the Interrater reliability of the scoring, a second rater (a mathematics education doctoral student) scored the pre-and posttest responses of 10 randomly selected participants. The coefficients for the Interrater reliability of scoring were .93 and .95 on the pre-and posttest respectively. Additionally, all students’ responses on the intervention sessions were examined to identify the features of their interactions with VMs that enabled them to obtain the most gains on the pre-posttest items.
An independent t -test revealed that although both OVM and SVM groups improved their overall performance on the questions requiring different levels of cognitive demand from the pre- to posttest, there was no significant difference between their overall gain scores [ t (46) = .282 and p > .05]. OVM students, however, differed considerably from SVM students in their gains on the “doing mathematics” items that require exploring and understanding the nature of mathematical concepts, processes, or relationships (see Figure 2).
Figure 2 : Comparison of SVM and OVM groups’ gains on Doing Mathematics (DM) items.
To identify students’ performance on the three subsets of pre-and posttest items; procedures without connections, procedures with connections, and doing mathematics, a MANOVA test was conducted. There was only significant difference between OVM and SVM students’ gain scores on the “doing mathematics” items
[ F (1, 46) = 5.386, p < .05, partial η 2 = .105].
Although there are no significant differences between OVM and SVM groups’ gain scores throughout the “Procedures without Connections (PWOC)” and “Procedures with Connections (PC)” items, students in the
SVM group obtained higher gains than OVM students on these items (see Table 2).
M
PWOC
SD M
PC
SD M
DM
SD
OVM 2.12 2.11 1.41 1.96 1.02 1.25
SVM 3.02 2.00 2.54 2.84 .23 1.11
Table 2 : Comparison of mean gain scores between OVM and SVM groups.

The analysis of students’ responses in Intervention Session 1 involving the activities most related to the items that OVM students obtained the highest gains revealed that OVM students more often provided explanations and descriptions of their activities with the VMs in their answers than SVM group. For example, almost 70% of OVM students explained their answers in Session 1, however only about 40% of SVM students provided explanations in the session. In addition, the analysis also revealed that more OVM than SVM students used qualitative relations in their responses throughout the Session 1. For example, nearly 35% of OVM students mentioned about qualitative relationships, however none of SVM did in the session.
This study aimed to identify the effects of using VMs with two different instructional approaches on students’ solutions to the three types of problems requiring different levels of cognitive effort . The findings revealed that both open-ended exploratory activities and structured mathematics questions helped students’ improve their performance on the questions with different levels of cognitive demand. In particular, the instructional activities focusing on the exploratory learning activities (e.g., observing, interpreting) enabled students to demonstrate considerable improvement on their solutions to “Doing Mathematics (DM)” type of problems. In addition, students using VMs with open-ended exploratory activities more often referred to their interactions with the VMs to explain their solutions to posttest question than students in the SVM group.
On the other hand, instructional activities emphasizing the mathematical procedures and numerical values helped SVM students show higher performance (not significant) than OVM group on the PWOC and
PC type of items that require producing numeric answers and identifying the conceptual ideas that underlie procedures. Although SVM students had significantly lower gains than OVM group on the DM type of items involving higher-level demands, surprisingly they obtained higher gains than OVM group on the PC type of questions involving higher-level cognitive demands. One of the reasons of this interesting finding might be that
SVM students’ engagement with different procedures can enable them to identify the mathematical relations among the procedures, and thus helped them have higher gains on the PC questions. It is important to focus on lower-level tasks before moving to higher level tasks while designing instructional activities. As Gagné (1970) identified in his learning hierarchies, lower-level tasks are prerequisite to complete higher-level tasks.
These findings imply that teachers can incorporate VMs with various instructional activities to enhance their students’ performance on the mathematical problems that require lower or higher level cognitive demands. For example, once students have difficulties in using mathematical procedures, teachers can use VMs with procedural based instructional activities. Teachers can also integrate VMs into their instructional activities that focus on exploratory learning activities to help students improve their performance on the mathematical tasks involving higher-level cognitive demands.
As teachers become more aware of their students’ cognitive abilities, they can develop more effective instructional activities that initially require lower level cognitive processes and then gradually involve higher level cognitive processes (Tobias, 1982). As Stein et al. (2000) emphasized, different instructional tasks can be used to advance students’ learning of varied mathematics concepts. Thus, VMs used with diverse instructional approaches can enable students to improve their performance on both lower and higher level problems.
Clark, R. E. (1983). Reconsidering research on learning from media. Review of Educational Research , 53 (4),
445-45.
Crawford, C., & Brown, E. (2003). Integrating Internet-based mathematical manipulatives within a learning environment. Journal of Computers in Mathematics and Science Teaching, 22 (2), 169-180.
Gagné, R. M. (1970). The conditions of learning (2 nd ed.). New York: Holt, Rinehart & Winston.
Kaput, J.J. (1989). Linking representations in the symbolic systems of algebra. In S. Wagner & C. Kieran
(Eds.), Research agenda for mathematics education: Research issues in the learning and teaching of algebra (pp.167-194). Reston, VA: National Council of Teachers of Mathematics.

Kaput, J. J. (1992). Technology and mathematics education. In D. A. Grouws (Ed .), Handbook of research on mathematics teaching and learning (pp. 515-556). New York: Macmillan Publishing Company.
Leinhardt, G., Zaslavsky, O., & Stein, M. K. (1990). Functions, graphs and graphing: Tasks, learning and teaching. Review of Educational Research, 60 (1), 1-64.
Lobato, J., Ellis, A. B., & Muñoz, R. (2003). How "focusing phenomena" in the instructional environment support individual students' generalizations. Mathematical Thinking and Learning, 5 (1), 1-36.
McDermott, L. C., Rosenquist, M. L., & van Zee, E. H. (1987). Student difficulties in connecting graphs and physics: Examples from kinematics. American Journal of Physics, 55(6), 503-513.
Moyer-Packenham, P. S., Salkind, G., & Bolyard, J. J. (2008). Virtual Manipulatives Used by K-8 Teachers for
Mathematics Instruction : Considering Mathematical , Cognitive , and Pedagogical Fidelity.
Contemporary Issues in Technology and Teacher Education , 8 (3), 202–218. Retrieved from http://www.citejournal.org/vol8/iss3/mathematics/article1.cfm
National Council of Teachers of Mathematics (NCTM). Principles and Standards for School Mathematics.
Reston, VA: NCTM, 2000.
Resnick, M., Martin, F., Berg, R., Borovoy, R., Colella, V., Kramer, K., and Silverman, B. (1998). Digital
Manipulatives. Proceedings of the CHI '98 conference , Los Angeles, April 1998.
Scaife, M., & Rogers, Y. (1996). External cognition: How do graphical representations work? International
Journal of Humane- Computer Studies, 45 (2), 185-213.
Sinclair, M. (2006). Designing tasks with interactive geometry applets for use in research – Some methodological issues.
31-36.
The International Journal for Technology in Mathematics Education, 13 (1),
Smith, M.S., & Stein, M.K. (1998). Selecting and creating mathematical tasks: From research to practice.
Mathematics Teaching in the Middle School , 3 (5), 344-350.
Stein, M. K., Smith, M. S., Henningsen, M. A., & Silver, E. A. (2000). Implementing standards-based mathematics instruction: A casebook for professional development.
New York, NY: Teachers College
Press.
Stump, S. L. (2001). High school precalculus students understanding of slope as measure. School Science and
Mathematics, 101 (2 ), 81-89.
Suh, J., Moyer, P. S., & Heo, H. J. (2005). Examining technology uses in the classroom: Developing fraction sense using virtual manipulative concept tutorials. Journal of Interactive Online Learning, 3 (4), 1-21.
Suh, J., & Moyer, P. S. (2007). Developing students’ representational fluency using virtual and physical algebra balances. Journal of Computers in Mathematics and Science Teaching, 26 (2), 155-173.
Thompson, P. W. (1994). Images of rate and operational understanding of the fundamental theorem of calculus.
Educational Studies in Mathematics, 26 (2-3), 229-274.
Tobias, S. (1982). When do instructional methods. Educational Researcher , 11 (4), 4–9. doi:10.3102/0013189X011004004.
Zbiek, R. M., Heid, M. K., Blume, G. W., & Dick, T. P. (2007). Research on technology in mathematics education: The perspective of constructs. In F. Lester (Ed.), Handbook of research on mathematics teaching and learning (Vol. 2, pp. 1169-1207). Charlotte, NC:Information Age Publishing.