Navigation Maps and Problem Solving: revised 5/18/05
Running head: NAVIGATION MAPS AND PROBLEM SOLVING
The Effect of Navigation Maps on Problem Solving Tasks
Instantiated in a Computer-Based Video Game
A Dissertation
Submitted to: Dr. Harold O’Neil (Chair)
Dr. Edward Kazlauskas
Dr. Yanis Yortsos (Outside Member)
By
Richard Wainess
14009 Barner Ave.
Sylmar, CA 91342
(818) 364-9419
wainess@usc.edu
In fulfillment of Ph.D. in Educational Psychology and Technology
2005
1
Navigation Maps and Problem Solving: revised 5/18/05
Table of Contents
ABSTRACT ...............................................................................................................................6
CHAPTER I: INTRODUCTION ...............................................................................................7
Background of the Problem .......................................................................................................7
Statement of the Problem ...........................................................................................................9
Purpose of the Study ..................................................................................................................9
Significance of the Study .........................................................................................................10
Research Hypotheses and Questions .......................................................................................11
Overview of the Methodology .................................................................................................12
Assumptions.............................................................................................................................12
Limitations
..........................................................................................................................13
Delimitations ............................................................................................................................13
Organization of the Report.......................................................................................................14
CHAPTER II: LITERATURE REVIEW ................................................................................15
Cognitive Load Theory ............................................................................................................16
Working Memory .........................................................................................................17
Long Term Memory .....................................................................................................17
Schema Development ...................................................................................................18
Mental Models ..................................................................................................19
Elaboration and Reflection ...............................................................................20
Meaningful Learning ....................................................................................................20
Metacognition ...................................................................................................21
Cognitive Strategies ..............................................................................21
2
Navigation Maps and Problem Solving: revised 5/18/05
Mental Effort and Persistence...........................................................................22
Goals .................................................................................................................22
Goal Setting Theory ..............................................................................22
Goal Orientation Theory .......................................................................23
Self-Efficacy .....................................................................................................23
Self-Efficacy Theory.............................................................................24
Expectancy-Value Theory ....................................................................25
Task Value ............................................................................................25
Problem Solving ...............................................................................................26
O’Neil’s Problem Solving Model .........................................................27
Types of Cognitive Loads ............................................................................................28
Learner Control ............................................................................................................31
Summary of Cognitive Load ........................................................................................32
Games and Simulations............................................................................................................34
Games ...........................................................................................................................35
Simulations ...................................................................................................................36
Simulation-Games ........................................................................................................37
Motivational Aspects of Games ...................................................................................38
Fantasy ..............................................................................................................39
Control and Manipulation .................................................................................40
Challenge and Complexity ...............................................................................41
Curiosity ...........................................................................................................41
Competition ......................................................................................................42
3
Navigation Maps and Problem Solving: revised 5/18/05
Feedback ...........................................................................................................43
Fun ....................................................................................................................43
Learning and Other Outcomes for Games....................................................................45
Positive Outcomes from Games and Simulations ............................................45
Negative or Null Outcomes from Games and Simulations ..............................47
Relationship of Instructional Design to Effective Games and Simulations .....47
Reflection and Debriefing ................................................................................49
Summary of Games and Simulations ...........................................................................49
Assessment of Problem Solving ..............................................................................................52
Measurement of Content Understanding......................................................................52
Measurement of Problem Solving Strategies ............................................................56
Measurement of Self-Regulation ..............................................................................57
Summary of Problem Solving Assessment ..................................................................58
Scaffolding
..........................................................................................................................59
Graphical Scaffolding ..................................................................................................60
Navigation Maps...............................................................................................61
Contiguity Effect ..............................................................................................64
Split-Attention Effect .......................................................................................65
Summary of Scaffolding ..............................................................................................65
Summary of the Literature Review ..........................................................................................67
CHAPTER III: METHODOLOGY .........................................................................................72
Research Design.................................................................................................................72
Research Questions and Hypotheses .......................................................................................72
4
Navigation Maps and Problem Solving: revised 5/18/05
Sample
..........................................................................................................................73
Instruments ...............................................................................................................................73
Demographic, Game Preference, and Task Completion Questionnaire.......................74
Self-Regulation Questionnaire .....................................................................................74
SafeCracker ..................................................................................................................75
Navigation Map ............................................................................................................77
Knowledge Map ...........................................................................................................78
Content Understanding Measure ..................................................................................78
Domain-Specific Problem-Solving Strategies Measure ...............................................80
Procedure for the Study ...........................................................................................................81
Timing Chart ................................................................................................................82
Data Analysis ...........................................................................................................................83
REFERENCES ........................................................................................................................85
APPENDICES
Appendix A: Self-Regulation Questionnaire .............................................................104
Appendix B: SafeCracker® Expert Map ...................................................................106
Appendix C: Knowledge Map Specifications ............................................................107
5
Navigation Maps and Problem Solving: revised 5/18/05
6
ABSTRACT
Cognitive load theory defines a limited capacity working memory with associated auditory and
visual/spatial channels. Navigation in computer-based hypermedia and video game environments
is believed to place a heavy cognitive load on working memory. Current 3-dimensional
computer-based video games (3-D, computer-based video games) often include complex
occluded environments (conditions where vision is blocked by objects in the environment, such
as internal walls, trees, hills, or buildings) preventing players from plotting a direct visual course
from the start to finish locations. Navigation maps may provide the support needed to effectively
navigate in these environments. Navigation maps are a type of graphical scaffolding, and
scaffolding, including graphical scaffolding, helps learners by reducing the amount of cognitive
load placed on working memory. Navigation maps have been shown to be effective in 3-D,
occluded, video game environments requiring complex navigation and simple problem solving
tasks. Navigation maps have also been shown to be effective in 2-dimensional environments
involving complex problem solving tasks. This study will extend the research by combining
these two topics—navigation maps for navigation in 3-D, occluded, computer-based video
games and navigation maps in 2-dimensional environments with complex problem solving
tasks—by examining the effect of a navigation map on a 3-D, occluded, computer-based video
game problem solving task. In addition, the effect of a navigation map on motivation will be
examined.
Navigation Maps and Problem Solving: revised 5/18/05
7
CHAPTER I: INTRODUCTION
With the current power of computers and the current state-of-the-art of video games, it
is likely that future versions of educational video games will include immersive environments in
the form of 3-D, computer-based video games requiring navigation through occluded paths in
order to perform complex problem solving tasks. According to Cutmore, Hine, Maberly,
Langford, and Hawgood (2000), occlusion refers to conditions where vision is blocked by
objects in the environment, such as internal walls or large environmental features like trees, hills,
or buildings. Under these conditions, one cannot simply plot a direct visual course from the start
to finish locations (Cutmore et al., 2000). This study examines the use of navigation maps to
support navigation through a 3-D, occluded, computer-based video game involving a complex
problem-solving task.
Chapter one begins with an examination of the background of the problem. Next the
purpose of the study is discussed, followed by why the study is significant—how it will inform
the literature—and the hypotheses that will be addressed. The next sections in chapter one
include an overview of the methodology that will be utilized, assumptions that inform this topic,
study limitations and delimitations, and a brief explanation of the organization of this proposal.
Background of the Problem
Educators and trainers began to take notice of the power and potential of
computer games for education and training back in the 1970s and 1980s (Donchin, 1989;
Malone, 1981; Malone & Lepper, 1987; Ramsberger, Hopwood, Hargan, & Underfull, 1983;
Ruben, 1999; Thomas & Macredie, 1994). Computer games were hypothesized to be potentially
useful for instructional purposes and were also hypothesized to provide multiple benefits: (a)
complex and diverse approaches to learning processes and outcomes; (b) interactivity; (c) ability
Navigation Maps and Problem Solving: revised 5/18/05
8
to address cognitive as well as affective learning issues; and perhaps most importantly, (d)
motivation for learning (O’Neil, Baker, & Fisher, ,2002).
Research into the effectiveness of games and simulations as educational media has
been met with mixed reviews (de Jong & van Joolingen, 1998; Garris, Ahlers, & Driskell, 2002).
It has been suggested that the lack of consensus can be attributed to weaknesses in instructional
strategies embedded in the media and issues related to cognitive load (Chalmers, 2003; Cutmore,
Hine, Maberly, Langford, & Hawgood, 2000; Lee, 1999; Thiagarajan, 1998; Wolfe, 1997).
Cognitive load theory suggests that learning involves the development of schemas (Atkinson,
Derry, Renkl, & Wortham, 2000), a process constrained by a limited working memory with
separate channels for auditory and visual/spatial stimuli (Brunken, Plass, & Leutner, 2003).
Further, cognitive load theory describes an unlimited capacity, long-term memory that can store
vast numbers of schemas (Mousavi, Low, & Sweller, 1995).
The inclusion of scaffolding, which provides support during schema development by
reducing the load in working memory, is a form of instructional design; more specifically, it is an
instructional strategy (Allen, 1997; Clark, 2001). For example, graphical scaffolding, which
involves the use of imagery-based aids, has been shown to be an effective support for
graphically-based learning environments, including video games (Benbasat & Todd, 1993;
Farrell & Moore, 2000-2001; Mayer, Mautone, & Prothero, 2002). Navigation maps, a particular
form of graphical scaffolding, have been shown to be an effective scaffold for navigation of a 3dimensional (3-D) virtual environment (Cutmore et al., 2000). Navigation maps have also been
shown to be an effective support for navigating and problem-solving in a 2-dimension (2-D)
hypermedia environment (Baylor, 2001; Chou, Lin, & Sun, 2000), which is made up of nodes of
information and links between the various nodes (Bowdish, & Lawless, 1997). What has not
Navigation Maps and Problem Solving: revised 5/18/05
9
been examined, and is the purpose of this study, is the effect of navigation maps, utilized for
navigation in a 3-D, occluded, computer-based video game, on outcomes in a complex problemsolving task.
Statement of the Problem
A major instructional issue in learning by doing within simulated environments
concerns the proper type of guidance, that is, how best to create cognitive apprenticeship (Mayer,
Mautone, & Prothero, 2002). A virtual environment creates a number of issues with regards to
learning. Problem-solving within a virtual environment involves not only the cognitive load
associated with the to-be-learned material (referred to as intrinsic cognitive load: Paas,
Tuovinen, Tabbers, Van Gerven, 2003), it also includes cognitive load related to the visual
nature of the environment (referred to as extraneous cognitive load: Brunken, Plass, & Leutner;
Harp & Mayer, 1998), as well as navigating within the environment—either germane cognitive
load or extraneous cognitive load, depending on the relationship of the navigation to the learning
task (Renkl, & Atkinson, 2003). An important goal of instructional design within these
immersive environments involves determining methods for reducing the extraneous cognitive
load and/or germane cognitive load, thereby providing more working memory capacity for
intrinsic cognitive load (Brunken et al., 2003). This study will examine the reduction of cognitive
load, by providing graphical scaffolding in the form of a navigation map, to determine if this can
result in better performance outcomes as reflected in retention and transfer (Paas et al., 2003) in a
game environment.
Purpose of the Study
The purpose of this study is to examine the effect of a navigation map on a complex
problem solving task in a 3-D, occluded, computer-based video game. The environment for this
Navigation Maps and Problem Solving: revised 5/18/05
10
study is the interior of a mansion as instantiated in the game SafeCracker® (Daydream
Interactive, 1995). The navigation map is a printed version of the floor plan of the first floor,
with relevant room information, such as the name of the room. The problem solving task
involves navigating through the environment to locate specific rooms, to find and acquire items
and information necessary to open safes located within the prescribed rooms, and ultimately, to
open the safes. With one group playing the game while using the navigation map and the other
group playing the game without aid of a navigation map, this study will exam differences in
problem solving outcomes informed by the problem solving model defined by O’Neil (1999).
Significance of the Study
Research has examined the use of navigation maps, a particular form of graphical
scaffolding, as navigation support for complex problem-solving tasks within a hypermedia
environment, with the navigation map providing an overview of the 2-dimension structure which
had been segmented into nodes (Chou, Lin, & Sun, 2000). Research has also examined the use of
navigation maps as a navigational tool in 3-D virtual environments, but has only examined the
effect of the navigation map on navigation (Cutmore, Hine, Maberly, Langford, & Hawgood,
2000) or during a complex navigation task that involved a very basic problem solving task;
finding a key along the path in order to open a door at the end of the path (Galimberti, Ignazi,
Vercesi, &Riva, 2001). Research has not combined these two research topics; it has not
examined the use of navigation maps in relationship to a complex problem-solving task that
involved navigation within a complex 3-D virtual environment.
While a number of studies on hypermedia environments have examined the issue of site
maps to aid in navigation of the various nodes for problem solving tasks (e.g., Chou & Lin,
1998), no study has looked at the effect of the use of two-dimensional topological maps (a floor
Navigation Maps and Problem Solving: revised 5/18/05
11
plan) for navigation within a 3-dimensional video game environment in relationship to complex
problem solving task. It is argued here that the role of the two navigation map types (site map
and topological floor plan) serve the same purpose in terms of cognitive load. However, it is also
argued here that the spatial aspect of the two learning environments differ significantly, placing a
larger load on the visual/spatial channel of working memory with a 3-D video game environment
as compared to a 2-D hypermedia environment, thereby leaving less working memory capacity
in the 3-D video game for visual stimuli; the navigation map.
As immersive 3-D video games are becoming more common as commercial video games,
it is likely they will also become more common as educational media. Therefore, the role of
navigation maps to reduce the load induced by navigation and, therefore, reduce burdens on
working memory, is an important issue for enhancing the effectiveness of games as educational
environments.
Research Question and Hypotheses
Research Question 1: Will the problem solving performance of participants who use a
navigation map in a 3-D, occluded, computer-based video game (i.e., SafeCracker®) be better
than the problem solving performance of those who do not use the map (the control group)?
Hypothesis 1: Navigation maps will produce a significant increase in content
understanding compared to the control group.
Hypothesis 2: Navigation maps will produce a significant increase in problem solving
strategy retention compared to the control group.
Hypothesis 3: Navigation maps will produce a significant increase in problem solving
strategy transfer compared to the control group.
Navigation Maps and Problem Solving: revised 5/18/05
12
Hypothesis 4: There will be no significant difference in self-regulation between the
navigation map group and the control group. However, it is expected that higher levels of selfregulation will be associated with better performance.
Research Question 2: Will the continued motivation of participants who use a
navigation map in a 3-D, occluded, computer-based video game (i.e., SafeCracker®) be greater
than the continued motivation of those who do not use the map (the control group)?
Hypothesis 5: Navigation maps will produce a significantly greater amount of optional
continued game play compared to the control group.
Overview of the Methodology
The design of this study is an experimental with pre-, intermediate-, and post-tests for one
treatment group and one control group. Subjects will be randomly assigned to either the
treatment or the control group. Group sessions will involve only one group type: either all
treatment participants or all control participants. The experimental design involves
administration of pretest instruments, the treatment, administration of intermediate test
instruments, the treatment, and administration of the posttest instruments. At the end of the
session, participants will be debriefed and will be then allowed to continue playing on their own
for up to 30 additional minutes (to examine continued motivation).
Assumptions
This research assumes the acceptance of the cognitive load theory, which describes a
limited working memory with separate auditory and visual-spatial channels, an unlimited longterm memory, and the existence of schema. Also assumed is the influence of scaffolding on
reducing cognitive load, by assisting in the development of schema and in distributing cognitive
load.
Navigation Maps and Problem Solving: revised 5/18/05
13
Limitations
There are a number of potential weaknesses to the current study. First, is the nature of
the main instrument; the game SafeCracker®. The game was created in 1995 and, by current
standards for game graphics and dynamics, is not a particularly engaging game. While it is
assumed this issue will not affect problem-solving assessments, it is assumed it will likely
influence motivational outcomes—specifically, continued motivation as assessed by the desire to
continue playing the game after the experimental procedure is completed.
Another weakness is the introduction of both the contiguity effect (Mayer, Moreno,
Boire, & Vagge, 1999; Mayer & Sims, 1994; Moreno & Mayer, 1999), in the form of spatial
contiguity, and the split-attention effect (Mayer & Moreno, 1998; Yeung, Jin, & Sweller, 1997)
into the study; for the treatment group. Because the game is visual and will seen on a computer
screen, and the navigation map (the floor plan) given to the treatment group, also visual but in
printed format, will be spatially separate from the screen, cognitive load issues as described by
both the contiguity effect and the split-attention effect will be imposed. This study will not
examine the impact of this added load, and how it might influence learning outcomes, possibly
offsetting the benefits of the graphical scaffolding (the navigation map).
Delimitations
All participants will be undergraduate students of a tier-1 university; one of the top one
hundred universities in the United States. This suggests that the sample will not easily generalize
to a larger population. Second, since participants are all volunteers, it is likely they enjoy playing
video games. Therefore, they represent only the video game playing population, and study results
cannot be generalized to those who are less inclined to play or to volunteer to play video games.
A third delimitation is the time period of the study; summer. During summer, a large portion of
Navigation Maps and Problem Solving: revised 5/18/05
14
the student population is not on campus. Therefore, those who respond to the flyer, or other
marketing means for soliciting study participants, will not be a true representation of the full
student population. Combined, these delimitations suggest that the sample enlisted for this study
will be bar generalization beyond a narrow population.
Organization of the Report
Chapter one provides an overview of the study with a brief introduction and background
for the topic, the problem being addressed, the significance of the study, the hypotheses that will
be tested, an overview of the methodology of the experiments, and assumptions, limitations, and
delimitations related to the study. Chapter two is the literature review of the domains that inform
the current research: cognitive load theory, games and simulations, assessment of problemsolving, and scaffolding. Chapter three describes the study’s methodology, with discussions of
the population, the sample, the study, the instruments, the procedures, and the proposed data
analysis methods.
Navigation Maps and Problem Solving: revised 5/18/05
15
CHAPTER II: LITERATURE REVIEW
The literature review includes information on four areas relevant to the research topic:
cognitive load theory, games and simulations, assessment of problem solving, and scaffolding.
The cognitive load section is comprised of an introduction to cognitive load, followed by
discussions of working and long-term memory, schema development, and mental models and the
role of reflection and elaboration. Next, under cognitive load theory, is a discussion of
meaningful learning, including the role of metacognition and cognitive strategies, mental effort
and persistence, goals and related theories, self-efficacy and several related theories and topics,
and problem solving, with a discussion of O’Neil’s Problem Solving model (O’Neil, 1999).
Types of cognitive load are then discussed—specifically, intrinsic cognitive load, germane
cognitive load, and extraneous cognitive load—as well as learner control as informed by
cognitive load theory. The cognitive load theory section ends with a summary of the section.
Following cognitive load theory is a discussion of games and simulations, beginning
with defining games, simulations, and simulation-games. Next the motivational aspects of games
is introduced with a discussion of the major characteristics of motivation: fantasy, control and
manipulation, challenge and complexity, curiosity, competition, feedback, and fun. The final
section under games and simulations is learning and other outcomes, which ends with a short
discussion of the role of reflection and debriefing. Last is a summary of games and simulations.
The third section of chapter two is the assessment of problem solving focused on the
three constructs established in O’Neil’s Problem Solving model (O’Neil, 1999): measurement of
content understanding, measurement of problem solving strategies, and measurement of selfregulation. The section ends in with a summary of problem solving assessment.
Navigation Maps and Problem Solving: revised 5/18/05
16
The fourth and final section, scaffolding, begins with a general discussion of scaffolding,
followed by a review of the literature on a type of scaffolding relevant to this study, graphical
scaffolding. Within graphical scaffolding, navigation maps are examined, along with the
relationship of the contiguity effect and the split attention effect to inclusion of a navigation map.
The section ends in with a summary of scaffolding. The chapter ends with a summary of chapter
two.
Cognitive Load Theory
Cognitive load theory, which began in the 1980s and underwent substantial development
and expansion in the 1990s (Paas, Renkl, & Sweller, 2003), is concerned with the development
of instructional methods aligned with the learners’ limited cognitive processing capacity, to
stimulate their ability to apply acquired knowledge and skills to new situations (i.e., transfer).
Brunken, Plass, and Leutner (2003) argued that cognitive load theory is based on several
assumptions regarding human cognitive architecture: the assumption of a virtually unlimited
capacity of long-term memory, schema theory of mental representations of knowledge, and
limited-processing capacity assumptions of working memory (Brunken et al., 2003). Cognition is
the intellectual processes through which information is obtained, represented mentally,
transformed, stored, retrieved, and used. cognitive load theory is based on the idea that a
cognitive architecture exists consisting of a limited working memory, with partly independent
processing units for visual-spatial and auditory-verbal information (Mayer & Moreno, 2003), and
these structures interact with a comparatively unlimited long-term memory (Mousavi, Low, &
Sweller, 1995).
Cognitive load is the total amount of mental activity imposed on working memory at an
instance in time (Chalmers, 2003; Cooper, 1998; Sweller and Chandler, 1994, Yeung, 1999).
Navigation Maps and Problem Solving: revised 5/18/05
17
Researchers have proposed that working memory limitations can have an adverse effect on
learning (Sweller and Chandler, 1994, Yeung, 1999). According to Paas, Tuovinen, Tabbers, &
Van Gerven, (2003), cognitive load can be defined as a multidimensional construct representing
the load that performing a particular task imposes on the learner’s cognitive system. The
construct has a causal dimension reflecting the interaction between task and learner
characteristics, and an assessment dimension reflecting the measurable concepts of mental load,
mental effort, and performance (Paas et al., 2003). Cognitive load is a theoretical construct,
describing the internal processes of information processing that cannot be observed directly
(Brunken et al., 2003).
Working Memory
Working memory refers to the limited capacity for holding information in mind for
several seconds in the context of cognitive activity (Gevins et al., 1998). According to Brunken
et al. (2003), the Baddeley (1986) model of working memory assumes the existence of a central
executive that coordinates two slave systems, a visuospatial sketchpad for visuospatial
information such as written text or pictures, and a phonological loop for phonological
information such as spoken text or music (Baddeley, 1986, Baddeley & Logie, 1999). Both slave
systems are limited in capacity and independent from one another so that the processing
capacities of one system cannot compensate for lack of capacity in the other (Brunken et al.,
2003).
Long-Term Memory
According to Paas et al. (2003), working memory, in which all conscious cognitive
processing occurs, can handle only a very limited number of novel interacting elements; possibly
no more than two or three. In contrast, long-term memory has an unlimited, permanent capacity
Navigation Maps and Problem Solving: revised 5/18/05
18
(Tennyson & Breuer, 2002) and can contain vast numbers of schemas—cognitive constructs that
incorporate multiple elements of information into a single element with a specific function (Paas
et al., 2003). Noyes and Garland (2003) contended that information that is not held in working
memory will need to be retained by the long-term memory system. Storing more knowledge in
long-term memory reduces the load on working memory, which results in a greater capacity
being made available for active processing.
According to cognitive load theory, multiple elements of information can be chunked as
single elements in cognitive schema (Chalmers, 2003), and through repeated use can become
automated. Automated information, developed over hundreds of hours of practice (Clark, 1999),
can be processed without conscious effort, bypass working memory during mental processing,
thereby circumventing the limitations of working memory (Clark 1999; Mousavi et al., 1995).
Consequently, the primay goals of instruction are the construction (chunking) and automation of
schemas (Paas et al., 2003).
Schema Development
Schema is defined as a cognitive construct that permits people to treat multiple subelements of information as a single element, categorized according to the manner in which it will
be used (Kalyuga, Chandler, & Sweller, 1998). Schemas are generally thought of as ways of
viewing the world and, in a more specific sense, ways of incorporating instruction into our
cognition. Schema acquisition is a primary learning mechanism. Piaget proposed that learning is
the result of forming new schemas and building upon previous schemas (as cited in Chalmers,
2003). Schemas have the functions of storing information in long-term memory and of reducing
working memory load by permitting people to treat multiple elements of information as a single
element (Kalyuga, et al., 1998; Mousavi et al., 1995).
Navigation Maps and Problem Solving: revised 5/18/05
19
With schema use, a single element in working memory might consist of a large number
of lower level, interacting elements which, if processed individually, might have exceeded the
capacity of working memory (Paas et al., 2003). If a schema can be brought into working
memory in automated form, it will make limited demands on working memory resources,
leaving more resources available to search for a possible solution problem (Kalyuga et al., 1998).
Controlled use of schemas requires conscious effort, and therefore, working memory resources.
However, after being sufficiently practiced, schemas can operate under automatic, rather than
controlled, processing. Automatic processing of schemas requires minimal working memory
resources and allows for problem solving to proceed with minimal effort (Kalyuga, Ayers,
Chandler, & Sweller, 2003; Kalyuga et al., 1998; Paas et al., 2003).
Mental Models
Mental models explain human understanding of external reality, translating reality into
internal representations and utilizing them in problem solving (Park & Gittelman, 1995).
According to Allen (1997), mental models are usually considered the way in which people model
processes. This emphasis on process distinguishes mental models from other types of cognitive
organizers such as schemas. A mental model synthesizes several steps of a process and organizes
them as a unit. A mental model does not have to represent all of the steps which compose the
actual process (Allen, 1997). Mental models may be incomplete and may even be internally
inconsistent. Models of mental models are termed conceptual models. Conceptual models
include: metaphor; surrogates; mapping, task-action grammars, and plans. Mental model
formation depends heavily on the conceptualizations that individuals bring to a task (Park &
Gittelman, 1995).
Navigation Maps and Problem Solving: revised 5/18/05
20
Elaboration and Reflection
Elaboration and reflection are processes involved to the development of schemas and
mental models. Elaborations are used to develop schemas whereby nonarbitrary relations are
established between new information elements and the learner’s prior knowledge (van
Merrienboer, Kirshner, & Kester, 2003). Elaboration consists of the creation of a semantic event
that includes the to-be-learned items in an interaction (Kees & Davies, 1990). With reflection,
learners are encouraged to consider their problem-solving process and to try to identify ways of
improving it (Atkinson, Renkl, & Merrill, 2003). Reflection is reasoned and conceptual, allowing
the thinker to consider various alternatives (Howland, Laffey, & Espinosa, 1997). According to
Chi (2000) the self-explanation effect (aka reflection or elaboration) is a dual process that
involves generating inferences and repairing the learner’s own mental model.
Meaningful Learning
Meaningful learning is defined as deep understanding of the material, which includes
attending to important aspects of the presented material, mentally organizing it into a coherent
cognitive structure, and integrating it with relevant existing knowledge (Mayer & Moreno,
2003). Meaningful learning is reflected in the ability to apply what was taught to new situations;
problem solving transfer. Meaningful learning results in an understanding of the basic concepts
of the new material through its integration with existing knowledge (Davis, & Wiedenbeck,
2001).
According to assimilation theory, there are two kinds of learning: rote learning and
meaningful learning. Rote learning occurs through repetition and memorization. It can lead to
successful performance in situations identical or very similar to those in which a skill was
initially learned. However, skills gained through rote learning are not easily extensible to other
Navigation Maps and Problem Solving: revised 5/18/05
21
situations, because they are not based on deep understanding of the material learned. Meaningful
learning, on the other hand, equips the learner for problem solving and extension of learned
concepts to situations different from the context in which the skill was initially learned (Davis, &
Wiedenbeck, 2001; Mayer, 1981).Meaningful learning takes place when the learner draws
connections between the new material to be learned and related knowledge already in long-term
memory, known as the assimilative context (Ausubel, 1963; Davis, & Wiedenbeck, 2001).
Meaningful learning requires mental effort. According to Clark (2003b), “mindful” mental effort
requires instructional messages (feedback) that point out the novel elements of the to-be-learned
material and must emphasize the need to work hard. Instructional messages must present be
concrete and challenging, yet achievable, learning and performance goals. The following
sections address the majority of these topics.
Metacognition
Metacognition, or the management of cognitive processes, involves goal-setting,
strategy selection, attention, and goal checking (Jones, Farquhar, & Surry, 1995). According to
Harp and Mayer (1998), many cognitive models include the executive processes of selecting,
organizing, and integrating. Selecting involves paying attention to the relevant pieces of
information. Organizing involves building internal connections among the selected pieces of
information, such as causal chains. Integrating involves building external connections between
the incoming information and prior knowledge existing in the learner’s long-term memory (Harp
& Mayer, 1998).
Cognitive strategies. Cognitive strategies include rehearsal strategies, elaboration
strategies, organization strategies, affective strategies, and comprehension monitoring strategies.
These strategies are cognitive events that describe the way in which we process information
Navigation Maps and Problem Solving: revised 5/18/05
22
(Jones et al., 1995). Metacognition is a type of cognitive strategy that has executive control over
other cognitive strategies. Prior experience in solving similar tasks and using various strategies
will affect the selection of a cognitive strategy (Jones et al., 1995).
Mental Effort and Persistence
Mental effort is the aspect of cognitive load that refers to the cognitive capacity that is
actually allocated to accommodate the demands imposed by a task; thus, it can be considered to
reflect the actual cognitive load. Mental effort, relevant to the task and material, appears to be the
feature that distinguishes between mindless or shallow processing on the one hand, and mindful
or deep processing, on the other. Little effort is expended when processing is carried out
automatically or mindlessly (Salomon, 1983). Motivation generates the mental effort that drives
us to apply our knowledge and skills. According to Clark (2003d), “Without motivation, even the
most capable person will not work hard” (p. 21). However, mental effort investment and
motivation should not be equated. Motivation is the driving force, but for learning to actually
take place, some specific relevant mental activity needs to be activated. This activity is assumed
to be the employment of nonautomatic effortful elaborations (Salomon, 1983).
Goals
Motivation influences both attention and maintenance processes (Tennyson & Breuer,
2002), generating the mental effort that drives us to apply our knowledge and skills. Easy goals
are not motivating (Clark, 2003d). Additionally, it has been shown that individuals without
specific goals (such as “do your best”), do not work as long as those with specific goals, such as
“list 70 contemporary authors” (Thompson et al., 2002; Locke & Latham, 2003).
Goal setting theory, according to Thompson et al. (2002), is based on the simple
premise that people exert effort toward accomplishing goals. Goals may increase performance as
Navigation Maps and Problem Solving: revised 5/18/05
23
long as a few factors are taken into account, such as acceptance of the goal, feedback on progress
toward the goal, a goal that is appropriately challenging, and a goal that is specific (Thompson et
al., 2002). Goal setting guides the cognitive strategies in a certain direction. Goal checking are
those monitoring processes that check to see if the goal has been accomplished, or if the selected
strategy is working as expected. The monitoring process is active throughout an activity and
constantly evaluates the success of other processes. If a cognitive strategy appears not to be
working, an alternative may then be selected (Jones et al., 1995).
Goal orientation theory is concerned with the prediction that those with high
performance goals and a perception of high ability will exert great effort, and those with low
ability perceptions will avoid effort (Miller et al., 1996). Once we are committed to a goal, we
must make a plan to achieve the goal. A key element of all goal-directed planning is our personal
assessment of the necessary skills and knowledge required to achieve a goal. A key aspect of
self-efficacy assessment is our perception of how novel and difficult the goal is to achieve. The
ongoing results of this analysis are hypothesized to determine how much effort we will invest in
a goal (Clark, 1999).
Self-Efficacy
A number of items affect motivation and mental effort. In an extensive review of
motivation theories, Eccles and Wigfield (2002) discuss Brokowski and colleagues’ motivation
model that highlights the interaction of the following cognitive, motivational, and self-processes:
knowledge of oneself (including goals and self perceptions); domain-specific knowledge;
strategy knowledge; and personal-motivational states (including attributional beliefs, selfefficacy, and intrinsic motivation). In a study of college freshmen, Livengood (1992) found that
psychological variables (i.e., effort/ability reasoning, goal choice, and confidence) are strongly
Navigation Maps and Problem Solving: revised 5/18/05
24
associated with academic participation and satisfaction. And Corno and Mandinah (1983)
commented that students in classrooms actively engage in a variety of cognitive interpretations
of their environments and themselves which, in turn, influence the amount and kind of effort
they will expend on classroom tasks.
According to Clark (1999), the more novel the goal is perceived to be, the more effort
we will invest until we believe we might fail. At the point where failure expectations begin,
effort is reduced as we “unchoose” the goal to avoid a loss of control. This inverted U
relationship suggests that mental effort problems include two broad forms: over confidence and
under confidence (Clark, 1999). Therefore, the level of mental effort necessary to achieve goals
can be influenced by adjusting perceptions of goal novelty and the effectiveness of the strategies
people use to achieve goals (Clark, 1999).
Self-efficacy is defined as one’s belief about one’s ability to successfully carry out
particular behaviors (Davis, & Wiedenbeck, 2001). Perceived self-efficacy refers to subjective
judgments of how well one can execute a particular course of action, handle a situation, learn a
new skill or unit of knowledge, and the like (Salomon, 1983). Perceived self-efficacy has much
to do with how a class of stimuli is perceived. The more demanding the stimuli is perceived to
be, the less efficacious the perceiver would feel about it. Conversely, the more familiar, easy, or
shallow it is perceived, the more efficacious the perceiver would feel about handling it. It follows
that perceived self efficacy should be related to the perception of demand characteristics (the
latter includes the perceived worthwhileness of expending effort), and that both should affect
effort investment jointly (Salomon, 1983).
Self-efficacy theory. Self-efficacy theory predicts that students work harder on a
learning task when they judge themselves as capable versus when they lack confidence in their
Navigation Maps and Problem Solving: revised 5/18/05
25
ability to learn. Self-efficacy theory also predicts that students understand the material better
when they have high self-efficacy than when they have low self-efficacy (Mayer, 1998). Effort is
primarily influenced by specific and detailed self efficacy assessments of the knowledge required
to achieve tasks (Clark, 1999). A person’s belief about whether he or she has the skills required
to succeed at a task is possibly the most important factor in the quality and quantity of mental
effort that person will invest (Clark, 2003d).
Expectancy-Value Theory. Related to self-efficacy theories, expectancy-value theories
propose that the probability of behavior depends on the value of the goal and the expectancy of
obtaining that goal (Coffin & MacIntyre, 1999). Expectancies refer to beliefs about how we will
do on different tasks or activities, and values have to do with incentives or reasons for doing the
activity (Eccles & Wigfield, 2002). From the perspective of expectancy-value theory, goal
hierarchies (the importance and the order of goals) also could be organized around aspects of
task value. Different goals may be perceived as more or less useful, or more or less interesting.
Eccles and Wigfield (2002) suggest that the relative value attached to the goal should influence
its placement in a goal hierarchy, as well as the likelihood a person will try to attain the goal and
therefore exert mental effort. Clark (2003b) commented that the more instruction supports a
student’s interest and utility value for instructional goals, as well as the student’s self-efficacy for
the course, the more likely the student will become actively engaged in the instruction and persist
when faced with distractions.
Task value. Task value refers to an individual’s perceptions of how interesting,
important, and useful a task is (Coffin & MacIntyre, 1999). Interest in, and perceived importance
and usefulness of, a task comprise important dimensions of task value (Bong, 2001). Citing
Eccles’ expectancy-value model, Townsend and Hicks (1997) stated that the perception of task
Navigation Maps and Problem Solving: revised 5/18/05
26
value is affected by a number of factors, including the intrinsic value of a task, its perceived
utility value, and its attainment value. Thus, engagement in an academic task may occur because
of interest in the task, or because the task is required for advancement in some other area
(Townsend & Hicks, 1997). According to Corno and Mandinah (1983), a task linked to one’s
aspirations (a “self-relevant” task) is a key condition for task value (Corno & Mandinah, 1983).
Problem-Solving
Problem solving is the intellectual skill to propose solutions to previously unencountered
problem situations (Tennyson & Breuer, 2002). A problem exists when a problem solver has a
goal but does not know how to reach it, so problem solving is mental activity aimed at finding a
solution to a problem (Baker & Mayer, 1999). Problem solving is associated with situations
dealing with previously unencountered problems, requiring the integration of knowledge to form
new knowledge (Tennyson & Breuer, 2002). A first condition of problem solving involves the
differentiation process of selecting knowledge that is currently in storage using known criteria.
Concurrently, this selected knowledge is integrated to form a new knowledge. Cognitive
complexity within this condition focuses on elaborating the existing knowledge base (Tennyson
& Breuer, 2002). Problem solving may also involve situations requiring the construction of
knowledge by employing the entire cognitive system. Therefore, the sophistication of a proposed
solution is a factor of the person’s knowledge base, level of cognitive complexity, higher-order
thinking strategies, and intelligence (Tennyson & Breuer, 2002). According to Mayer (1998),
successful problem solving depends on three components—skill, metaskill, and will—and each
of these components can be influenced by instruction. Metacognition—in the form of
metaskill—is central in problem solving because it manages and coordinates the other
components (Mayer, 1998).
Navigation Maps and Problem Solving: revised 5/18/05
27
O’Neil’s Problem Solving model. O’Neil’s Problem Solving model (O’Neil, 1999,
see figure 1 below) is based on Mayer and Wittrock’s (1996) conceptualization: “Problem
solving is cognitive processing directed at achieving a goal when no solution method is obvious
to the problem solver” (p. 47). This definition is further analyzed into components suggested by
the expertise literature: content understanding or domain knowledge, domain-specific problemsolving strategies, and self-regulation (see, e.g., O’Neil, 1999, 2002). Self-regulation is
composed of metacognition (planning and self-checking) and motivation (effort and selfefficacy). Thus, in the specifications for the construct of problem solving, to be a successful
problem solver, “one must know something (content knowledge), possess intellectual tricks
(problem-solving strategies), be able to plan and monitor one’s progress towards solving the
problem (metacognition), and be motivated to perform” (effort and self-efficacy; O’Neil, 1999,
pp. 255-256).
Figure 1. O’Neil’s Problem Solving Model
Problem Solving
Content
Understanding
Problem-Solving
Strategies
Self-Regulation
Metacognition
Planning
Domain
Specific
Domain
Independent
SelfMonitoring
Motivation
Effort
SelfEfficacy
Navigation Maps and Problem Solving: revised 5/18/05
28
In problem solving, the skeletal structures are instantiated in content domains, so that a
set of structurally similar models for thinking about problem solving is applied to science,
mathematics, and social studies. These models may vary in the explicitness of problem
representations, the guidance about strategy (if any), the demands of prior knowledge, the focus
on correct procedures, the focus on convergent or divergent responses, and so on (Baker &
Mayer, 1999). Domain-specific aspects of problem solving (e.g., the part that is unique to
geometry, geology, or genealogy) involve the specific content knowledge, the specific
procedural knowledge in the domain, any domain-specific cognitive strategies (e.g., geometric
proof, test, and fix), and domain specific discourse (O’Neil, 1998, as cited in Baker & Mayer,
1999). Both domain-independent and domain-dependent knowledge are usually essential for
problem solving. Domain-dependent analyses focus on the subject matter as the source of all
needed information (Baker & O’Neil, 2002).
Types of Cognitive Load
Cognitive load researchers have identified up to three types of cognitive load. All agree
on intrinsic cognitive load (Brunken et al., 2003; Paas et al., 2003; Renkl, & Atkinson, 2003),
which is the load involved in the process of learning; the load required by metacognition,
working memory, and long-term memory. Another load agreed upon is extraneous cognitive
load. However, it is the scope of this load that is in dispute. To some researchers, any cognitive
load that is not intrinsic cognitive load is extraneous cognitive load. To other researchers, nonintrinsic cognitive load is divided into germane cognitive load and extraneous cognitive load.
Germane cognitive load is the cognitive load required to process the intrinsic cognitive load
(Renkl, & Atkinson, 2003). From a non-computer-based perspective, this could include
searching a book or organizing notes, in order to process the to-be-learned information. From a
Navigation Maps and Problem Solving: revised 5/18/05
29
computer-based perspective, this could include the interface and controls a learner must interact
with in order to be exposed to, and process, the to-be-learned material. In contrast to germane
cognitive load, these researchers see extraneous cognitive load as the load caused by any
unnecessary stimuli, such as fancy interface designs or extraneous sounds (Brunken et al., 2003).
For each of the two working memory subsystems (visual/spatial, and auditory/verbal),
the total amount of cognitive load for a particular individual under particular conditions is
defined as the sum of intrinsic, extraneous, and germane load induced by the instructional
materials. Therefore, a high cognitive load can be a result of a high intrinsic cognitive load (i.e.,
the nature of the instructional content itself). It can, however, also be a result of a high germane
cognitive load (i.e., a result of activities performed on the materials that result in a high memory
load) or high extraneous cognitive load (i.e., a result of inclusion of unnecessary information or
stimuli that result in a high memory load; Brunken et al., 2003).
Low-element interactivity refers to environments where each element can be learned
independently of the other elements, and there is little direct interaction between the elements.
High-element interactivity refers to environments where there is so much interaction between
elements that they cannot be understood until all the elements and their interactions are
processed simultaneously. As a consequence, high-element interactivity material is difficult to
understand (Paas et al., 2003). Element interactivity is the driver of intrinsic cognitive load,
because the demands on working memory capacity imposed by element interactivity are intrinsic
to the material being learned. Reduction in intrinsic load can occur only by dividing the material
into small learning modules (Paas et al., 2003).
Germane or effective cognitive load is influenced by the instructional design. The
manner in which information is presented to learners and the learning activities required of
Navigation Maps and Problem Solving: revised 5/18/05
30
learners are factors relevant to levels of germane cognitive load. Whereas extraneous cognitive
load interferes with learning, germane cognitive load enhances learning (Renkl, & Atkinson,
2003).
Extraneous cognitive load (Renkl, & Atkinson, 2003) is the most controllable load, since
it is caused by materials that are unnecessary to instruction. However, those same materials may
be important for motivation. Unnecessary items are globally referred to as extraneous. However,
another category of extraneous items, seductive details (Mayer, Heiser, & Lonn, 2001), refers to
highly interesting but unimportant elements or instructional segments. These segments usually
contain information that is tangential to the main themes of a story, but are memorable because
they deal with controversial or sensational topics (Schraw, 1998). The seductive detail effect is
the reduction of retention caused by the inclusion of extraneous details (Harp & Mayer, 1998)
and affects both retention and transfer (Moreno & Mayer, 2000).
Complicating the issue of seductive details is the arousal theory which suggests that
adding entertaining auditory adjuncts will make a learning task more interesting, because it
creates a greater level of attention so that more material is processed by the learner (Moreno &
Mayer, 2000). A possible solution is to leave the seductive details, but guide the learner away
from them and to the relevant information (Harp & Mayer, 1998).
While attempting to focus on a mental activity, most of us, at one time or another, have
had our attention drawn to extraneous sounds (Banbury, Macken, Tremblay, & Jones, 2001). On
the surface, seductive details and auditory adjuncts (such as sound effects or music) seem
similar. However, the underlying cognitive mechanisms are quire different. Whereas seductive
details seem to prime inappropriate schemas into which incoming information is assimilated,
auditory adjuncts seem to overload auditory working memory (Moreno & Mayer, 2000).
Navigation Maps and Problem Solving: revised 5/18/05
31
According to Brunken et al. (2003), both extraneous and germane cognitive load can be
manipulated by the instructional design of the learning material (Brunken et al., 2003).
Learner Control
In contrast to more traditional technologies that only deliver information, computerized
learning environments offer greater opportunities for interactivity and learner control. These
environments can offer simple sequencing and pace control, or they can allow the learner to
decide which, and in what order, information will be accessed (Barab, Young, & Wang, 1999).
The term navigation refers to a process of tracking one’s position in an environment, whether
physical or virtual, to arrive at a desire destination. A route through the environment consists of
either a series of locations or a continuous movement along a path (Cutmore et al., 2000).
Effective navigation of a familiar environment depends upon a number of cognitive factors.
These include working memory for recent information, attention to important cues for location,
bearing and motion, and finally, a cognitive representation of the environment which becomes
part of a long-term memory, a cognitive map (Cutmore et al., 2000).
Hypermedia environments divide information into a network of multimedia nodes
connected by various links (Barab, Bowdish, & Lawless, 1997). According to Chalmers (2003),
how easily learners become disoriented in a hypermedia environment may be a function of the
user interface. One area where disorientation can be a problem is in the use of links. Although
links create the advantage of exploration, there is always the chance learners may become lost,
not knowing where they were, where they are going, or where they are (Chalmers, 2003). In a
virtual 3-D environment, Cutmore et al. (2000) argue that navigation becomes problematic when
the whole path cannot be viewed at once and is largely occluded by objects in the environment.
Navigation Maps and Problem Solving: revised 5/18/05
32
Under these conditions, one cannot simply plot a direct visual course from the start to finish
locations. Rather, knowledge of the layout of the space is required (Cutmore et al., 2000).
Message complexity, stimulus features, and additional cognitive demands inherent in
hypermedia, such as learner control, may combine to exceed the cognitive resources of some
learners (Daniels & Moore, 2000). Dillon and Gabbard (1998) found that novice and lower
aptitude students have the greatest difficulty with hypermedia. Children are particularly
susceptible to the cognitive demands of interactive computer environments. According to
Howland, Laffey, and Espinosa (1997), many educators believe that young children do not have
the cognitive capacity to interact and make sense of the symbolic representations of computer
environments.
In spite of the intuitive and theoretical appeal of hypertext environments, empirical
findings yield mixed results with respect to the learning benefits of learner control over program
control of instruction (Niemiec, Sikorski, & Wallberg, 1996; Steinberg, 1989). And six extensive
meta-analyses of distance and media learning studies in the past decade have found the same
negative or weak results (Bernard, et al, 2003). In reference to distance learning environments,
Clark (2003c) argued that when sequencing, contingencies, and learning strategies permit only
minimal learner control over pacing, then “except for the most advanced expert learners, learning
will be increased” (p. 14).
Summary of Cognitive Load
Cognitive Load Theory is based on the assumptions of a limited working memory with
separate channels for auditory and visual/spatial stimuli, and a virtually unlimited capacity longterm memory that stores schemas of varying complexity and level of automation (Brunken et al.,
2003). According to Paas et al. (2003), cognitive load refers to the amount of load placed on
Navigation Maps and Problem Solving: revised 5/18/05
33
working memory. Cognitive load can be reduced through effective use of the auditory and
visual/spatial channels, as well as schemas stored in long-term memory. Cognitive load also
reflects the measurable concepts of mental load, mental, effort and performance (Paas et al,
2003).
Meaningful learning is defined as deep understanding of the material and is reflected in
the ability to apply what was taught to new situations; i.e., problem solving transfer. (Mayer &
Moreno, 2003). Meaningful learning requires effective metacognitive skills: the management of
cognitive processes (Jones, Farquhar, & Surry, 1995), including selecting relevant information,
organizing connections among the pieces of information, and integrating (i.e., building) external
connections between incoming information and prior knowledge that exists in long-term memory
(Harp & Mayer, 1998). Mental effort refers to the cognitive capacity allocated to a task. Mental
effort is affected by motivation, and motivation cannot exist without goals (Clark, 2003d). Goals
are further affected by self-efficacy, the belief in one’s ability to successfully carry out a
particular behavior (Davis & Wiedenbeck, 2001).
Problem solving is “cognitive processing directed at transforming a given situation into
a desired situation when no obvious methods of solution is available to the problem solver”
(Baker & Mayer, 1999, p. 272). O’Neil’s Problem Solving model (O’Neil, 1999) defines three
core constructs of problem solving: content understanding, problem solving strategies, and selfregulation. Each of these components is further defined with subcomponents. There are three
types of cognitive load that can be defined in relationship to a learning or problem solving task:
intrinsic cognitive load, germane cognitive load, and extraneous cognitive load.
Intrinsic cognitive load refers to the cognitive load placed on working memory by the
to-be-learned material (Paas et al., 2003). Germane cognitive load refers to the cognitive load
Navigation Maps and Problem Solving: revised 5/18/05
34
required to access and process the intrinsic cognitive load. For example, the problem solving
processes that are instantiated in the learning process so that learning can occur (Renkl &
Atkinson, 2003). Extraneous cognitive load refers to the cognitive load imposed by stimuli that
neither support the learning process (i.e., germane cognitive load) nor are part of the to-belearned material (i.e., intrinsic cognitive load). A specific form of extraneous cognitive load is
seductive details; highly interesting but unimportant elements or instructional segment, that are
often used to provide memorable or engaging experiences (Mayer et al., 2001; Schraw, 1998).
An important goal of instructional design is to balance intrinsic, germane, and extraneous
cognitive loads to support learning outcomes, and to recognize that the specific balance is
dependent on a number of factors (Brunen et al., 2003), including the amount of prior knowledge
and the need for motivation.
Learner control, which is inherent in interactive computer-based media, allows for
control of pacing and sequencing (Barab, Young, & Wang, 1999). It also provides an opportunity
for cognitive overload in the form of disorientation; loss of place (Chalmers, 2003). Further,
Daniels and Moore (2000) argued that message complexity, stimulus features, and additional
cognitive demands inherent in hypermedia (e.g., learner control) may combine to exceed the
cognitive resources of some learners. Further, learner control is a potential source for extraneous
cognitive load. Ultimately, these issues may be the cause of mixed reviews of learner control
(Bernard, et al, 2003; Niemiec, Sikorski, & Wallberg, 1996; Steinberg, 1989).
GAMES AND SIMULATIONS
According to Ricci, Salas, and Cannon-Bowers (1996), “computer-based educational
games generally fall into one of two categories: simulation games and video games. Simulation
games model a process or mechanism relating task-relevant input changes to outcomes in a
Navigation Maps and Problem Solving: revised 5/18/05
35
simplified reality that may not have a definite endpoint” (p. 296). Ricci et al. further comment
that simulation games “often depend on learners reaching conclusions through exploration of the
relation between input changes and subsequent outcomes” (p. 296). Video games, on the other
hand, are competitive interactions bound by rules to achieve specified goals that are dependent
on skill or knowledge and often involve chance and imaginary settings (Randel, Morris, Wetzel,
& Whitehill, 1992).
One of the first problems areas with research into games and simulations is terminology.
Many studies that claim to have examined the use of games did not use a game (e.g., Santos,
2002). At best, they used an interactive multimedia that exhibits some of the features of a game,
but not enough features to actually be called a game. A similar problem occurs with simulations.
A large number of research studies use simulations but call them games (e.g., Mayer et al.,
2002). Because the goals and features of games and simulations differ, it is important when
examining the potential effects of the two media to be clear about which one is being examined.
However, there is little consensus in the education and training literature on how games and
simulations are defined.
Games
According to Garris, Ahlers, and Driskell (2002) early work in defining games suggested
that there are no properties that are common to all games and that games belong to the same
semantic category only because they bear a family resemblance to one another. Betz (1995-1996)
argued that a game is being played when the actions of individuals are determined by both their
own actions and the actions of one or more actors.
A number of researchers agree that games have rules (Crookall, Oxford, & Saunders,
1987; Dempsey, Haynes, Lucassen, & Casey, 2002; Garris et al., 2002; Ricci, 1994).
Navigation Maps and Problem Solving: revised 5/18/05
36
Researchers also agree that games have goals and strategies to achieve those goals (Crookall &
Arai, 1995; Crookall et al. 1987; Garris et al., 2002; Ricci, 1994). Many researchers also agree
that games have competition (e.g., Dempsey et al., 2002) and consequences such as winning or
losing (Crookall et al., 1987; Dempsey et al., 2002).
Betz (1995-1996) further argued that games simulate whole systems, not parts, forcing
players to organize and integrate many skills. Students will learn from whole systems by their
individual actions, individual action being the student’s game moves. Crookall et al. (1987) also
noted that a game does not intend to represent any real-world system; it is a “real” system in its
own right. According to Duke (1995), games are situation specific. If well designed for a specific
situation or condition, the same game should not be expected to perform well in a different
environment.
Simulations
In contrast to games, Crookall and Saunders (1989) viewed a simulation as a
representation of some real-world system that can also take on some aspects of reality. Similarly,
Garris et al. (2002) wrote that key features of simulations are they represent real-world systems,
and Henderson, Klemes, and Eshet (2000) commented that a simulation attempts to faithfully
mimic an imaginary or real environment that cannot be experienced directly, for such reasons as
cost, danger, accessibility, or time. Berson (1996) also argued that simulations allow access to
activities that would otherwise be too expensive, dangerous, or impractical for a classroom. Lee
(1999) added that a simulation is defined as a computer program that relates elements together
through cause and effect relationships.
Thiagarajan (1998) argued that simulations do not reflect reality; they reflect someone’s
model of reality. According to Thiagarajan, a simulation is a representation of the features and
Navigation Maps and Problem Solving: revised 5/18/05
37
behaviors of one system through the use of another. At the risk of introducing a bit more
ambiguity, Garris et al. (2002) proposed that simulations can contain game features, which leads
to the final definition: simulation-games.
Simulation-Games
Garris et al. (2002) argued that it is possible to consider games and simulations as similar
in some respects, keeping in mind the key distinction that simulations propose to represent
reality and games do not. Combining the features of the two media, Rosenorn and Kofoed (1998)
described simulation/gaming as a learning environment where participants are actively involved
in experiments, for example, in the form of role-plays, or simulations of daily work situations, or
developmental scenarios.
This paper will use the definitions of games, simulations, and simulation-games as
defined by Gredler (1996), which combine the most common features cited by the various
researchers, and yet provide clear distinctions between the three media. According to Gredler,
Games consist of rules that describe allowable player moves, game
constraints and privileges (such as ways of earning extra turns), and penalties
for illegal (nonpermissable) actions. Further, the rules may be imaginative in
that they need not relate to real-world events (p. 523).
This definition is in contrast to a simulation, which Gredler (1996) defines as “a dynamic
set of relationships among several variables that (1) change over time and (2) reflect authentic
causal processes” (p. 523). In addition, Gredler describes games as linear and simulations as nonlinear, and games as having a goal of winning while simulations have a goal of discovering
causal relationships. Gredler also defines a mixed metaphor referred to as simulation games or
Navigation Maps and Problem Solving: revised 5/18/05
38
gaming simulations, which is a blend of the features of the two interactive media: games and
simulations.
Motivational Aspects of Games
According to Garris et al. (2002), motivated learners are easy to describe; they are
enthusiastic, focused and engaged, they are interested in and enjoy what they are doing, they try
hard, and they persist over time. Furthermore, they are self-determined and driven by their own
volition rather than external forces (Garris et al., 2002). Ricci et al. (1996) defined motivation as
“the direction, intensity, and persistence of attentional effort invested by the trainee toward
training” (p. 297). Similarly, according to Malouf (1987-1988), continuing motivation is defined
as returning to a task or a behavior without apparent external pressure to do so when other
appealing behaviors are available. And more simply, Story and Sullivan (1986) commented that
the most common measure of continuing motivation is whether a student returns to the same task
at a later time.
With regard to video games, Asakawa and Gilbert (2003) argued that, without sources of
motivation, players often lose interest and drop out of a game. However, there seems little
agreement among researchers as to what those sources are—the specific set of elements or
characteristics that lead to motivation in any learning environment, and particularly with
educational games. According to Rieber (1996) and McGrenere (1996), motivational researchers
have offered the following characteristics as common to all intrinsically motivating learning
environments: challenge, curiosity, fantasy, and control (Davis & Wiedenbeck, 2001; Lepper &
Malone, 1987; Malone, 1981; Malone & Lepper, 1987). Malone (1981) and others also included
fun as a criteria for motivation.
Navigation Maps and Problem Solving: revised 5/18/05
39
For interactive games, Stewart (1997) added the motivational importance of goals and
outcomes. Locke and Latham (1990) also commented on the robust findings with regards to
goals and performance outcomes. They argued that clear, specific goals allow the individual to
perceive goal-feedback discrepancies, which are seen as crucial in triggering greater attention
and motivation. Clark (2001) further argued that motivation cannot exist without goals. The
following sections will focus on fantasy, control and manipulation, challenge and complexity,
curiosity, competition, feedback, and fun. The role of goals was discussed previously in this
proposal in fostering effort and motivation was discussed earlier in this document.
Fantasy
Research suggests that material may be learned more readily when presented in an
imagined context that interests the learner than when presented in a generic or decontextualized
form (Garris et al., 2002). Malone and Lepper (1987) defined fantasy as an environment that
evokes “mental images of physical or social situations that do not exist” (p. 250). Rieber (1996)
commented that fantasy is used to encourage learners to imagine they are completing an activity
in a context in which they are really not present. However, Rieber described two types of
fantasies: endogenous and exogenous. Endogenous fantasy weaves relevant fantasy into a game,
while exogenous simply sugar coat a learning environment with fantasy. An example of an
endogenous fantasy would be the use of a laboratory environment to learn chemistry, since this
environment is consistent with the domain. An example of an exogenous environment would be
a using a hangman game to learn spelling, because hanging a person has nothing to do with
spelling. Rieber (1996) noted that endogenous fantasy, not exogenous fantasy, is important to
intrinsic motivation, yet exogenous fantasies are a common and popular element of many
educational games.
Navigation Maps and Problem Solving: revised 5/18/05
40
According to Malone and Lepper (1987), fantasies can offer analogies or metaphors for
real-world processes that allow the user to experience phenomena from varied perspectives. A
number of researchers (Anderson and Pickett, 1978; Ausubal, 1963; Malone and Lepper, 1978;
Malone and Lepper, 1987; Singer, 1973) argued that fantasies in the form of metaphors and
analogies provide learners with better understanding by allowing them to relate new information
to existing knowledge. According to Davis and Wiedenbeck (2001), metaphor also helps learners
to feel directly involved with objects in the domain so the computer and interface become
invisible.
Control and Manipulation
Hannifin and Sullivan (1996) define control as the exercise of authority or the ability to
regulate, direct, or command something. Control, or self-determination, promotes intrinsic
motivation because learners are given a sense of control over the choices of actions they may
take (deCharms, 1986; Deci, 1975; Lepper & Greene, 1978). Furthermore, control implies that
outcomes depend on learners’ choices and, therefore, learners should be able to produce
significant effects through their own actions (Davis, & Wiedenbeck, 2001). According to Garris
et al. (2002), games evoke a sense of personal control when users are allowed to select strategies,
manage the direction of activities, and make decisions that directly affect outcomes, even if those
actions are not instructionally relevant.
However, Hannafin & Sullivan (1996) warned that research comparing the effects of
instructional programs that control all elements of the instruction (program control) and
instructional programs in which the learner has control over elements of the instructional
program (learner control) on learning achievement has yielded mixed results. Dillon and
Gabbard (1998) commented that novice and lower aptitude students have greater difficulty when
Navigation Maps and Problem Solving: revised 5/18/05
41
given control, compared to experts and higher aptitude students, and Niemiec, Sikorski, and
Walberg (1996) argued that control does not appear to offer any special benefits for any type of
learning or under any type of condition.
Challenge and complexity
Challenge, also referred to as effectance, competence, or mastery motivation (Bandura,
1977; Csikszentmihalyi, 1975; Deci, 1975; Harter, 1978; White, 1959), embodies the idea that
intrinsic motivation occurs when there is a match between a task and the learner’s skills. The
task should not be too easy or too hard, because in either case, the learner will lose interest
(Clark, 1999; Malone & Lepper, 1987). Clark (1999) describes this effect as an inverted Ushaped relationship with lack of effort existing on either side of difficultly ranging from too easy
to too hard. Stewart (1997) similarly commented that games that are too easy will be dismissed
quickly. According to Garris et al. (2002), there are several ways in which an optimal level of
challenge can be obtained. Goals should be clearly specified, yet the probability of obtaining that
goal should be uncertain, and goals must also be meaningful to the individual. Garris and
colleagues argued that linking activities to valued personal competencies, embedding activities
within absorbing fantasy scenarios, or engaging competitive or cooperative motivations could
serve to make goals meaningful.
Curiosity
According to Rieber (1996), challenge and curiosity are intertwined. Curiosity arises from
situations in which there is complexity, incongruity, and discrepancy (Davis & Wiedenbeck,
2001). Sensory curiosity is the interest evoked by novel situations and cognitive curiosity is
evoked by the desire for knowledge (Garris et al. 2002). Cognitive curiosity motivates the learner
to attempt to resolve the inconsistency through exploration (Davis, & Wiedenbeck, 2001).
Navigation Maps and Problem Solving: revised 5/18/05
42
Curiosity is identified in games by unusual visual or auditory effects, and by paradoxes,
incompleteness, and potential simplifications (Westbrook & Braithwaite, 2002). Curiosity is the
desire to acquire more information, which is a primary component of the players’ motivation to
learn how to operate the game (Westbrook & Braithwaite, 2001).
Malone and Lepper (1987) noted that curiosity is one of the primary factors that drive
learning and is related to the concept of mystery. Garris et al. (2002) commented that curiosity is
internal, residing in the individual, and mystery is an external feature of the game itself. Thus,
mystery evokes curiosity in the individual, and this leads to the question of what constitutes
mystery (Garris et al. 2002). Research suggests that mystery is enhanced by incongruity of
information, complexity, novelty, surprise, and violation of expectations (Berlyne, 1960),
incompatibility between ideas and inability to predict the future (Kagan, 1972), and information
that is incomplete and inconsistent (Malone & Lepper, 1987).
Competition
Studies on competition with games and simulations have mixed results, due to
preferences and reward structures. A study by Porter, Bird, and Wunder (1990-1991) examining
competition and reward structures found that the greatest effects of reward structure were seen in
the performance of those with the most pronounced attitudes toward either competition or
cooperation. The results also suggested that performance was better when the reward structure
matched the individual’s preference. According to the authors, implications are that emphasis on
competition will enhance the performance of some learners but will inhibit the performance of
others (Porter et al., 1990-1991).
Yu (2001) investigated the relative effectiveness of cooperation with and without intergroup competition in promoting student performance, attitudes, and perceptions toward subject
matter studied, computers, and interpersonal context. With fifth-graders as participants, Yu
Navigation Maps and Problem Solving: revised 5/18/05
43
found that cooperation without inter-group competition resulted in better attitudes toward the
subject matter studies and promoted more positive inter-personal relationships both within and
among the learning groups, as compared to competition (Yu, 2001). The exchange of ideas and
information both within and among the learning groups also tended to be more effective and
efficient when cooperation did not take place in the context of inter-group competition (Yu,
2001).
Feedback
Feedback within games can be provided for learners to quickly evaluate their progress
against the established game goal. This feedback can take many forms, such as textual, visual,
and aural (Rieber, 1996). According to Ricci et al. (1996), within the computer-based game
environment, feedback is provided in various forms including audio cues, score, and remediation
immediately following performance. The researchers argued that these feedback attributes can
produce significant differences in learner attitudes, resulting in increased attention to the learning
environment. Clark (2003a) argued that, for feedback to be effective, it must be based on
“concrete learning goals that are clearly understood” (p. 18) and that it describe the gap between
the learner’s current performance and the goal. Additionally, the feedback must not be focused
on the failure to achieve the goal (Clark, 2003a).
Fun
Quinn (1994, 1997) argued that for games to benefit educational practice and learning,
they need to combine fun elements with aspects of instructional design and system design that
include motivational, learning, and interactive components. According to Malone (1981) three
elements (fantasy, curiosity, and challenge) contribute to the fun in games. While fun has been
cited as important for motivation and, ultimately, for learning, there is little empirical evidence
Navigation Maps and Problem Solving: revised 5/18/05
44
supporting the concept of fun. It is possible that fun is not a construct but, rather, represents other
concepts or constructs. Relevant alternative concepts or constructs include play, engagement, and
flow.
Play is entertainment without fear of present or future consequences; it is fun (Resnick &
Sherer, 1994). According to Rieber, Smith, and Noah (1998), play describes an intense learning
experience in which both adults and children voluntarily devote enormous amounts of time,
energy, and commitment and, at the same time, derive great enjoyment from the experience; this
is termed serious play (Rieber et al., 1998). Webster et al. (1993) found that labeling software
training as play showed improved motivation and performance. According to Rieber and Matzko
(2001), serious play is an example of an optimal life experience.
Csikszentmihalyi (1975; 1990) defines an optimal experience as one in which a person is
so involved in an activity that nothing else seems to matter; termed flow or a flow experience.
When completely absorbed in and activity, he or she is ‘carried by the flow,’ hence the origin of
the theory’s name (Rieber and Matzko, 2001). Rieber and Matzko (2001) offered a broader
definition of flow, commenting that a person may be considered in flow during an activity when
experiencing one or more of the following characteristics: Hours pass with little notice;
challenge is optimized; feelings of self-consciousness disappear; the activity’s goals and
feedback are clear; attention is completely absorbed in the activity; one feels in control; and one
feels freed from other worries (Rieber & Matzko, 2001). According to Davis and Wiedenbeck
(2001), an activity that is highly intrinsically motivating can become all-encompassing to the
extent that the individual experiences a sense of total involvement, losing track of time, space,
and other events. Davis and Wiedenbeck also argued that the interaction style of a software
package is expected to have a significant effect on intensity of flow. It should be noted that
Navigation Maps and Problem Solving: revised 5/18/05
45
Rieber and Matzko (2001) have contended that play and flow differ in one respect; learning is an
expressed outcome of serious play but not of flow.
Engagement is defined as a feeling of directly working on the objects of interest in the
worlds rather than on surrogates. According to Davis and Wiedenbeck (2001), this interaction or
engagement can be used along with the components of Malone and Lepper’s (1987) intrinsic
motivation model to explain the effect of an interaction style on intrinsic motivation, or flow.
Garris et al. (2002) commented that training professionals are interested in the intensity of
involvement and engagement that computer games can invoke, to harness the motivational
properties of computer games to enhance learning and accomplish instructional objectives.
Learning and Other Outcomes for Games
Results from studies reporting on the performance and learning outcomes from games are
mixed. This section is subdivided into four discussions. First will be a discussion of studies
indicating positive results regarding performance and learning outcomes attributed to games and
simulations. Second will be a discussion of studies indicating negative or null results regarding
performance and learning outcomes attributed to games and simulations. Third will be a
discussion of the relationship of instructional design to effectiveness of educational games and
simulations as an explanation of mixed results from game and simulation studies. Last will be a
discussion of reflection and debriefing as a necessary component to learning, with specific
references to the learning instantiated in games and simulations.
Positive Outcomes from Games and Simulations
Simulations and games have been cited as beneficial for a number of disciplines and for a
number of educational and training situations, including aviation training (Salas, Bowers, &
Rhodenizer, 1998), aviation crew resource management (Baker, Prince, Shrestha, Oser, & Salas,
Navigation Maps and Problem Solving: revised 5/18/05
46
1993), military mission preparation (Spiker & Nullmeyer, n.d.), laboratory simulation (Betz,
1995-1996), chemistry and physics education (Khoo & Koh, 1998), urban geography and
planning (Adams, 1998; Betz, 1995-1996), farm and ranch management (Cross, 1993), language
training (Hubbard, 1991), disaster management (Stolk, Alexandrian, Gros, & Paggio, 2001), and
medicine and health care (Westbrook & Braithwaite, 2001; Yair, Mintz, & Litvak, 2001). For
business, games and simulations have been cited as useful for teaching strategic planning
(Washburn & Gosen, 2001; Wolfe & Roge, 1997), finance (Santos, 2002), portfolio management
(Brozik, & Zapalska, 2002), marketing (Washburn & Gosen), knowledge management
(Leemkuil, de Jong, de Hoog, & Christoph, 2003), and media buying (King & Morrison, 1998).
In addition to teaching domain-specific skills, games have been used to impart more
generalizable skills. Since the mid 1980s, a number of researchers have used the game Space
Fortress, a 2-D, simplistic arcade-style game, with a hexagonal “fortress” in the center of the
screen surrounded by two concentric hexagons, and a space ship, to improve spatial and motor
skills that transfer far outside gameplay, such as significantly improving the results of fighter
pilot training (Day, Arthur, and Gettman, 2001). Also, in a series of five experiments, Green and
Bavelier (2003) showed the potential of video games to significantly alter visual selection
attention. Similarly, Greenfield, DeWinstanley, Kilpatrick, & Kaye (1994) found, with
experiments involving college students, that video game practice could significantly alter the
participants’ strategies of spatial attentional deployment.
According to Ricci et al. (1996), results of their study provided evidence that computerbased gaming can enhance learning and retention of knowledge. They further commented that
positive trainee reaction might increase the likelihood of student involvement with training (i.e.,
devote extra time to training). In a more guarded position, Leemkuil et al. (2003) commented
Navigation Maps and Problem Solving: revised 5/18/05
47
that much of the work on the evaluation of games has been anecdotal, descriptive, or judgmental,
yet there are some indications that they are effective and superior to case studies in producing
knowledge gains, especially in the area of strategic management (Wolfe, 1997).
Negative or Null Outcomes from Games and Simulations
A number of researchers have addressed the issue of the motivational aspects of games,
arguing that the motivation attributed to enjoyment of educational games may not necessarily
indicate learning and, possibly, might indicate less learning. Garris et al. (2002) noted that,
although students generally seem to prefer games over other, more traditional, classroom training
media, reviews have reported mixed results regarding the training effectiveness of games.
Druckman (1995) concluded that games seem to be effective in enhancing motivation and
increasing student interest in subject matter, yet the extent to which that translates into more
effective learning is less clear. With caution, Brougere (1999) commented that anything that
contributes to the increase of emotion (such as the quality of the design of video games)
reinforces the attraction of the game but not necessarily its educational interest. Similarly, Salas
et al. (1998) commented that liking a simulation does not necessarily transfer to learning.
Salomon (1984) went even further, by commenting that a more positive attitude can actually
indicate less learning. And in an early meta-analysis of the effectiveness of simulation games,
Dekkers and Donatti (1981) found a negative relationship between duration of training and
training effectiveness. Simulation games became less effective the longer the game was used
(suggesting that perhaps trainees became bored over time).
Relationship of Instructional Design to Effective Games and Simulations
de Jong and van Joolingen (1998), after reviewing a large number of studies on learning
from simulations, concluded, “there is no clear and univocal outcome in favor of simulations. An
Navigation Maps and Problem Solving: revised 5/18/05
48
explanation why simulation based learning does not improve learning results can be found in the
intrinsic problems that learners may have with discovering learning” (p. 181). These problems
are related to processes such as hypothesis generation, design of experiments, interpretation of
data, and regulation of learning. After analyzing a large number of studies, de Jong and van
Joolingen (1998) concluded that adding instructional support to simulations might help to
improve the situation.
The generally accepted position is that games themselves are not sufficient for learning but
there are elements in games that can be activated within an instructional context that may
enhance the learning process (Garris et al., 2002). In other words, outcomes are affected by the
instructional strategies employed (Wolfe, 1997). Leemkuil et al. (2003), too, commented that
there is general consensus that learning with interactive environments such as games,
simulations, and adventures is not effective when no instructional measure or support is added.
According to Thiagarajan (1998), if not embedded with sound instructional design, games
and simulations often end up truncated exercises often mislabeled as simulations. Gredler (1996)
further commented that poorly developed exercises are not effective in achieving the objectives
for which simulations are most appropriate—that of developing students’ problem-solving skills.
Lee (1999) commented that effect size never tells us under what conditions students learn more,
less, or not at all compared with the comparison group. For instructional prescription, we need
information dealing with instructional variables, such as instructional mode, instructional
sequence, knowledge domain, and learner characteristics (Lee, 1999).
Navigation Maps and Problem Solving: revised 5/18/05
49
Reflection and Debriefing
Instructional strategies that researchers have suggested as beneficial to learning from
games and simulations are reflection and debriefing. Brougere (1999) argued that a game cannot
be designed to directly provide learning. A moment of reflexivity is required to make transfer
and learning possible. Games require reflection, which enables the shift from play to learning.
Therefore, debriefing (or after action review), which includes reflection, appears to be an
essential contribution to research on play and gaming in education (Brougere, 1999; Leemkuil et
al., 2003; Thiagarajan, 1998). According to Garris et al. (2002), debriefing is the review and
analysis of events that occurred in the game. Debriefing provides a link between what is
represented in the simulation or gaming experience and the real world. It allows the learners to
draw parallels between game events and real-world events. Debriefing allows learners to
transform game events into learning experiences. Debriefing may include a description of events
that occurred in the game, analysis of why they occurred, and the discussion of mistakes and
corrective actions. Garris et al. (2002) argued that learning by doing must be coupled with the
opportunity to reflect and abstract relevant information for effective learning to occur.
Summary of games and simulation section.
Computer-based educational games fall into three categories: games, simulations, and
simulation games. Games consist of rules, can contain imaginative contexts, are primarily linear,
and include goals as well as competition, either against other players or against a computer
(Gredler, 1996). Simulations display the dynamic relationship among variables which change
over time and reflect authentic causal processes. Simulations are non-linear and have a goal of
discovering causal relationships through manipulation of independent variables. Simulation
games are a blend of games and simulations (Gredler, 1996).
Navigation Maps and Problem Solving: revised 5/18/05
50
Clark (2003d) argued that mental effort is affected by motivation. Beginning with the
work of Malone (1981), a number of constructs have been described as providing the
motivational aspects of games: fantasy, control and manipulation, challenge and complexity,
curiosity, competition, feedback, and fun. Fantasy is defined as an environment that evokes
“mental images of physical or social situations that do not exist” (Malone & Lepper, 1987, p.
250). Malone & Lepper (1987) also commented that fantasies can offer analogies and metaphors,
and Davis and Wiedenbeck (2001) argued that metaphors can help learners feel more directly
involved in the domain.
Control and manipulation promote intrinsic motivation, because learners are given a
sense of control over their choices and actions (deCharms, 1986, Deci, 1975). Challenge
embodies the idea that intrinsic motivation occurs when there is a match between a task and the
learner’s skills (Bandura, 1977, Csikszentmihalyi, 1975; Harter, 1978). The task should be
neither too hard nor too easy, otherwise, in both cases, the learner would lose interest (Clark,
1999; Malone & Lepper, 1987). According to Rieber (1996), curiosity and challenge are
intertwined. Curiosity arises from situations in which there is complexity, incongruity, and
discrepancy (Davis & Wiedenbeck, 2001) and Malone and Lepper (1997) argued that curiosity is
one of the primary factors that drive learning.
While Malone (1981) defines competition as important to motivation, studies on
competition with games and simulations have resulted in mixed findings, due to individual
learner preferences, as well as the types of reward structures connected to the competition (e.g.,
Porter, Bird, & Wunder, 1990-1991; Yu, 2001). Another motivational factor in games, feedback,
allows learners to quickly evaluate their progress and can take many forms, such as textual,
visual, and aural (Rieber, 1996). Ricci et al. (1996) argued that feedback can produce significant
Navigation Maps and Problem Solving: revised 5/18/05
51
differences in learner attitudes, resulting in increased attention to a learning environment.
However, Clark (2003a) commented that feedback must be focused on clear learning goals and
current performance results.
The last category contributing to motivation, fun, is possibly an erroneous category.
Little empirical evidence exists for the construct. However, evidence does support the related
constructs of play, engagement, and flow. Play is entertainment without fear of present of future
consequences (Resnick & Sherer, 1994). Webster et al. (1993) found that labeling software
training as play improved motivation and performance. Csikszentmihalyi (1975; 1990) defines
flow as an optimal experience in which a person is so involved in an activity that nothing else
seems to matter. According to Davis and Wiedenbeck (2001), engagement is the feeling of
working directly on the objects of interest in a world, and Garris et al. (2002) argued that
engagement can harness the motivational properties of computer games to enhance learning and
accomplish instructional objectives.
While numerous studies have cited the learning benefits of games and simulations
(e.g., Adams, 1998; Baker et al., 1997; Betz, 1995-1996; Khoo & Koh, 1998), others have found
mixed, negative, or null outcomes from games and simulations, specifically in relationship to the
of enjoyment of a game to learning from the game (e.g., Brougere, 1999; Dekkers & Donatti,
1981; Druckman, 1995). There appears to be consensus among a large number of researcher with
regards to the negative, mixed, or null findings, suggesting that the cause might be a lack of
sound instructional design embedded in the games (de Jong & van Joolingen, 1998; Garris et al.,
2002; Gredler, 1996; Lee, 1999; Leemkuil et al., 2003; Thiagarajan, 1998; Wolfe, 1997).
Among the various instructional strategies, reflection and debriefing have been cited
as critical to learning with games and simulations. Brougere (1999) argued that games cannot be
Navigation Maps and Problem Solving: revised 5/18/05
52
designed to directly provide learning; reflection is required to make transfer and learning
possible. Debriefing provides an opportunity for reflection (Brougere, 1999, Garris et al., 2002;
Thiagarajan, 1998).
ASSESSMENT OF PROBLEM SOLVING
According to O’Neil’s Problem Solving model (1999), successful problem solving
requires content understanding, problem solving strategies, and self-regulation. Therefore, proper
assessment of problem solving should address all three constructs.
Measurement of Content Understanding
According to Mayer and Moreno (2003), meaningful learning is reflected in retention
and transfer. Transfer refers to the ability to apply what was taught to new situations. According
to Day et al. (2001), declarative knowledge, which is an indication of retention, reflects the
amount of knowledge or facts learned. Similarly, Davis and Wiedenbeck (20010 commented that
meaningful learning results in an understanding of the basic concepts of the new material
through its integration with existing knowledge. Mayer and Moreno (1998) assessed content
understanding with retention and transfer questions. In their study on the split-attention effect in
multimedia learning, they used a retention test and a matching test containing a series of
questions to assess the extent to which participants retained knowledge delivered by the
multimedia.
Day et al. (2001), proposed knowledge maps as an alternative method to measure content
understanding. A knowledge map is a structural representation that consists of nodes and links.
Each node represents a concept in the domain of knowledge. Each link, which connects two
nodes, represents the relationship between the nodes; that is, the relationship between the two
concepts (Schau & Mattern, 1997). Knowledge structures are based on the premise that people
Navigation Maps and Problem Solving: revised 5/18/05
53
organize information into patterns that reflect the relationships which exist between concepts and
the features that define them (Day et al., 2001). Day et al. further argued that, in contrast to
declarative knowledge which reflects the amount of knowledge or facts learned, knowledge
structures represent the organization of the knowledge.
As Schau and Mattern (1997) point out, learners should not only be aware of the concepts
but also of the connections among them. In a training context, knowledge structures reflect the
degree to which trainees have organized and comprehended the content of training (Day et al.,
2001). Knowledge maps, which are graphical representations of knowledge structures, have been
used as an effective tool to learn complex subjects (Herl et al., 1996) and to facilitate critical
thinking and (West, Pomeroy, Park, Gerstenberger, & Sandoval, 2000). Several studies also
revealed that knowledge maps are not only useful for learning, but are a reliable and efficient
measurement of content understanding, as well (Herl et al., 1999; Ruiz-Primo, Schultz, &
Shavelson, 1997). The results of a study by Day et al. (2001) indicated that knowledge
structures are predictive of both skill retention and skill transfer and can therefore be a viable
indices of training outcomes, and Ruiz-Primo et al. (1997) proposed a framework for
conceptualizing knowledge maps as a potential assessment tool in science, because it allows for
organization and discrimination between concepts.
Ruiz-Primo et al. (1997) claimed that as an assessment tool, knowledge maps are
identified as a combination of three components: (a) a task that allows a student to perform his or
her content understanding in the specific domain (b) a format for student’s responses, and (c) a
scoring system by which the student’s knowledge map could be accurately evaluated. Chuang
(2003) modified this framework to serve as an assessment specification using a concept map.
Researchers have successfully applied knowledge maps to measure students’ content
Navigation Maps and Problem Solving: revised 5/18/05
54
understanding in science for both high school students and adults (e.g., Chuang, 2003; Herl et al.,
1999; Schacter et al., 1999; Schau et al., 2001). For example, Schau et al. (2001) used select-andfill-in knowledge maps to measure secondary and postsecondary students’ content understanding
of science in two studies. The results of the participant’s performance on the knowledge maps
correlated significantly with that of a multiple choice test, a traditional measure of learning
(r= .77 for eighth grade and r=. 74 for seventh grade), providing validity to the use of knowledge
maps to assess learning outcomes. CRESST developed a computer-based knowledge mapping
system, which measures the deeper understanding of individual students and teams, reflects
thinking processes in real-time, and economically reports student thinking process data back to
teachers and student (Chung et al., 1999; O’Neil, 1999; Schacter et al., 1999). The computerbased knowledge map has been used successfully in a number of studies (e.g., Chuang, 2003;
Chung et al., 1999; Hsieh, 2001; Schacter et al., 1999).
In the four studies, the map contained 18 concepts of environmental science, and seven
links for relationships, such as “cause,” “influence,” and “used for.” Subjects were asked to
create a knowledge map in the computer-based environment. In the study conducted by Schacter
et al. (1999) students were evaluated by creating individual knowledge map, after searching a
simulated World Wide Web environment. In the studies conducted by Chung et al. (1999), Hsieh
(2001), and Chuang (2003) two students constructed a group map cooperatively through
networked computers. Results of the cooperative studies showed that using networked computers
to measure group processes was feasible. Figures 2 and 3 show the knowledge map used for the
three studies.
Navigation Maps and Problem Solving: revised 5/18/05
55
Figure 2: User Interface
As seen in Figure 2, the screen of computer was divided into three major parts. The
bottom section was for communication between the two team members. The middle section is
where the one of the team members constructed the knowledge map. The top section contains
four menu items: “Session,” “Add Concept,” “Available Links,” and “About.” Figure three
shows the drop-down menu that appears when “Add Concept” is clicked. Clicking when the
mouse pointer is over a concept adds that link to the knowledge map. Figure 2 shows three
concepts that were added: desk, safe, and key. Figure 2 also shows links that were added to the
concept map by (A) clicking on one concept on the screen, holding the shift-key down and
clicking a second concept, so that both concepts are “selected,” then clicking on the “Available
Links” menu and selection the appropriate link type from the drop-down menu that appeared.
Navigation Maps and Problem Solving: revised 5/18/05
56
Figure 3: Adding Concepts to the Knowledge Map
Measurement of Problem Solving Strategies
According to Baker and Mayer (1999), “Problem solving is cognitive processing directed
at transforming a given situation into a desired situation when no obvious method of solution is
available to the problem solver” (p. 272). Simply put, problem solving is mental activity aimed
at finding a solution to a problem. According to Baker and Maker, a promising direct approach
to knowledge representation, “more parsimonious than a traditional performance assessment,” is
knowledge or concept mapping, in which “the learner constructs a network consisting of nodes
(e.g., key words or terms) and links (e.g., ‘is part of’, ‘led to’, ‘is an example of’)” (p. 274).
Problem solving strategies can be categorized as domain-independent (-general) and
domain-dependent (-specific) (Alexander, 1992; Bruning, Schraw, & Ronning, 1999; O’Neil,
1999; Perkins & Salomon, 1989). Domain-specific knowledge is knowledge about a particular
Navigation Maps and Problem Solving: revised 5/18/05
57
field of study or a subject, such as the application of equations in a math question, the application
of a formula in a chemistry problem, or the specific strategies to be successful in a game.
Domain-general knowledge is the broad array of problem solving knowledge that is not linked
with a specific domain, such as the application of multiple representations and analogies in a
problem-solving task or the use of Boolean search strategies in a search task (Chuang, 2003).
Transfer questions have been examined as an alternative to performing transfer tasks. For
example, in a recent study, Mayer and Moreno (1998) assessed participants’ problem-solving
strategies with a list of transfer questions. Responses to the transfer questions were positively
correlated with performance as measured by retention and transfer, indicating that transfer
questions are a viable alternative to transitions methods of measuring retention and transfer, such
as tests and novel problem solving (Mayer & Moreno, 1998).
Measure of Self-Regulation
While Brunning, Schraw, and Ronning (1999), commented that some researchers believe
self-regulation includes three core components—metacognitive awareness, strategy use, and
motivational control—, according to O’Neil’s Problem Solving model (O’Neil, 1999), selfregulation is composed of only two core components: metacognition and motivation. Strategy
use is a separate construct that encompasses domain-specific and domain-general knowledge.
Within O’Neil’s model, metacognition encompasses two subcategories, planning and selfchecking/monitoring (Hong & O’Neil, 2001; O’Neil & Herl, 1998; Pintrich & DeGroot, 1990),
and motivation is indicated by mental effort and self-efficacy (Zimmerman, 1994; 2000).
O’Neil and Herl (1998) developed a trait self-regulation questionnaire examining the four
components of self-regulation (planning, self-checking/monitoring, mental effort, and selfefficacy). Of the four components, planning is the first step, since learners must have a plan to
Navigation Maps and Problem Solving: revised 5/18/05
58
achieve the proposed goal. Self-efficacy is one’s belief in his or her capability to accomplish a
task, and effort is amount of mental effort exerted on a task. In the trait self-regulation
questionnaire developed by O’Neil and Herl, planning, self-checking, self-efficacy, and effort
are assessed using eight questions each. The reliability of this self-regulation inventory has been
established in previous studies (e.g., Hong & O’Neil, 2001). For example, in the research
conducted by Hong and O’Neil (2001), the reliability estimates (coefficient α) of the four
subscales of self-regulation, planning, self-checking, effort, and self-efficacy were .76, .06, .83,
and .85 respectively; Research also provided evidence for construct validation. To evaluate
problem-solving ability, previous researchers (e.g., Baker & Mayer, 1999; Baker & O’Neil,
2002; Mayer, 2002; O’Neil, 1999) argued that computer-based assessment has the merit of
integrating validity to generate test items and the efficiency of computer technology as a means
of presenting and scoring tests.
Summary of Problem Solving Assessment
Problem solving is cognitive processing directed at achieving a goal when no solution
method is obvious to the problem solver (Baker & Mayer, 1999). In O’Neil’s Problem Solving
model (O’Neil, 1999), problem solving strategies can be categorized into two types: domainindependent (-general) and domain-dependent (-specific) problem solving strategies. Selfregulation includes two sub-categories: metacognition and motivation. Metacognition is
composed of self-checking/monitoring and planning. Motivation is comprised of effort and selfefficacy.
Knowledge maps are reliable and efficient for the measurement of content understanding.
CRESST has developed a simulated Internet Web space to evaluate problem solving strategies
such as information searching strategies and feedback inquiring strategies. Computer-based
Navigation Maps and Problem Solving: revised 5/18/05
59
problem-solving assessments are economical, efficient and valid measures that employ
contextualized problems that require students to think for extended periods of time and to
indicate the problem-solving heuristics they were using and why.
SCAFFOLDING
As discussed in earlier, cognitive load theory is concerned with methods for reducing the
amount of cognitive load placed on working memory during learning and problem solving
activities. Clark (2003b) commented that instructional methods must also keep the cognitive load
from instructional presentations to a minimum. Scaffolding is considered a viable instructional
method that assists in cognitive load reduction. There are a number of definitions of scaffolding
in the literature. Chalmers (2003) defines scaffolding as the process of forming and building
upon a schema (Chalmers, 2003). In a related definition, van Merrionboer, Kirshner, and Kester
(2003) defined the original meaning of scaffolding as all devices or strategies that support
students’ learning. More recently, van Merrienboer, Clark, and de Croock (2002) defined
scaffolding as the process diminishing support as learners acquire more expertise. Allen (1997)
defined scaffolding as the process of training a student on core concepts and then gradually
expanding the training. Ultimately, the core principle embodied in each of these definitions is
that scaffolding is concerned with the amount of cognitive load imposed by learning and
provides methods for reducing that load. Therefore, for the purposes of this review, all four
definitions of scaffolding will be considered.
As defined by Clark (2001), instructional methods are external representations of
internal cognitive processes that are necessary for learning but which learners cannot or will not
provide for themselves. They provide learning goals (e.g., demonstrations, simulations, and
analogies: Alessi, 2000; Clark 2001), monitoring (e.g., practice exercises: Clark, 2001), feedback
Navigation Maps and Problem Solving: revised 5/18/05
60
(Alessi, 2000; Clark 2001; Leemkuil, de Jong, de Hoog, & Christoph, 2003), and selection (e.g.,
highlighting information: Alessi, 2000; Clark, 2001). In addition, Alessi (2000) adds: giving
hints and prompts before student actions; providing coaching, advice, or help systems; and
providing dictionaries and glossaries. Jones et al. (1995) added advance organizers, graphical
representations of problems, and hierarchical knowledge structures. Each of these examples is a
form of scaffolding.
In learning by doing in a virtual environment, students can actively work in realistic
situations that simulate authentic tasks for a particular domain (Mayer et al., 2002). A major
instructional issue in learning by doing within simulated environments concerns the proper type
of guidance, that is, how best to create cognitive apprenticeship (Mayer et al. 2002). Mayer et al.
(2002) also commented that their research shows that discovery-based learning environments can
be converted into productive venues for learning when appropriate cognitive scaffolding is
provided; specifically, when the nature of the scaffolding is aligned with the nature of the task,
such as pictorial scaffolding for pictorially-based tasks and textual-based scaffolding for
textually-based tasks. For example, in a recent study, Mayer et al. (2002) found that students
learned better from a computer-based geology simulation when they were given some support
about how to visualize geological features, versus textual or auditory guidance.
Graphical Scaffolding
According to Allen (1997), selection of appropriate text and graphics can aid the
development of mental models, and Jones et al. (1995) commented that visual cues such as maps
and menus as advance organizers help learners conceptualize the organization of the information
in a program (Jones et al., 1995). A number of researchers support the use of maps as visual aids
Navigation Maps and Problem Solving: revised 5/18/05
61
and organizers (Benbasat & Todd, 1993: Chou & Lin, 1998; Chou, Lin, & Sun, 2000; Farrell &
Moore, 2000-2001; Ruddle et al, 1999)
Chalmers (2003) defines graphic organizers is organizers of information in a graphic
format, which act as spatial displays of information that can also act as study aids. Jones et al.
(1995) argued that interactive designers should provide users with visual or verbal cues to help
them navigate through unfamiliar territory. Overviews, menus, icons, or other interface design
elements within the program should serve as advance organizers for information contained in the
interactive program (Jones et al., 1995). In addition, the existence of bookmarks enables
recovering from the possibility of disorientation; loss of place (Dias, Gomes, & Correia, 1999).
However, providing such support devices does not guarantee learners will use them. For
example, in an experiment involving a virtual maze, Cutmore et al. (2000) found that, while
landmarks provided useful cues, males utilized them significantly more often than females did.
Navigation maps
Cutmore et al. (2000) define navigation as “…a process of tracking one’s position in a
physical environment to arrive at a desired destination” (p. 224). A route through the
environment consists of either a series of locations or a continuous movement along a path.
Cutmore et al. further commented that “Navigation becomes problematic when the whole path
cannot be viewed at once but is largely occluded by objects in the environment’” (p. 224). These
may include internal walls or large environmental features such as trees, hills, or buildings.
Under these conditions, one cannot simply plot a direct visual course from the start to finish
locations. Rather, knowledge of the layout of the space is required. Navigation maps or other
descriptive information may provide that knowledge (Cutmore et al. 2000).
Navigation Maps and Problem Solving: revised 5/18/05
62
Effective navigation of a familiar environment depends upon a number of cognitive
factors. These include working memory for recent information, attention to important cues for
location, bearing and motion, and finally, a cognitive representation of the environment which
becomes part of a long-term memory, a cognitive map (Cutmore et al., 2000). According to Yair,
Mintz, and Litvak (2001), the loss of orientation and “vertigo” feeling which often accompanies
learning in a virtual-environment is minimized by the display of a traditional, two-dimensional
dynamic map. The map helps to navigate and to orient the user, and facilitates an easier learning
experience. Dempsey et al. (2002) also commented that an overview of player position was
considered an important feature in adventure games.
A number of experiments have examined the use of navigation maps in virtual
environments. Chou and Lin (1998) and Chou et al. (2000) examined various navigation map
types, with some navigation maps offering global views of the environment and others offering
more localized views, based on the learner’s current location. In their experiments using over one
hundred college students, they found that any form of navigation map produced more efficient
navigation of the site as well as better development of cognitive maps (concept or knowledge
maps), compared to having no navigation map. Additionally, the global navigation map results
for navigation and concept map creation were significantly better than any of the local navigation
map variations or the lack of navigation map, while use of the local navigation maps was not
significantly better than not having a navigation map. This suggests that, while the use of
navigation maps is helpful, the nature or scope of the navigation map influences its effectiveness
(Chou & Lin, 1998).
According to Tkacz (1998), soldiers use navigation maps as tools, which involves
spatial reasoning, complex decision making, symbol interpretation, and spatial problem solving.
Navigation Maps and Problem Solving: revised 5/18/05
63
In her study, Tkacz (1998) examined the procedural components of cognitive maps required for
using and understanding topographic navigation maps, stating that navigation map interpretation
involves both top-down (retrieved from long-term memory) and bottom-up (retrieved from the
environment and the navigation map) procedures. Therefore, Tkacz examined the cognitive
components underlying navigation map interpretation to assess the influence of individual
differences on course success and on real world position location. In addition, Tkacz, related
position location ability to video game performance in a simulated environment. The learning
goal of the study was for 105 marines (non-random assignment) to learn to match their position
in the real world with a navigation map. From the results of the study, Tkacz argued that spatial
orientation is the most important cognitive component of terrain association and that orientation
and, to a lesser extent, reasoning ability are important in navigation map interpretation and
course performance. It should be noted that, while Tkacz referred to the instrument as a game, it
appears to fit the Gredler’s (1996) definition of a simulation, not a game or simulation game.
Mayer et al. (2002) commented that a major instructional issue in learning by doing
within simulated environments concerns the proper type of guidance, which they refer to as
cognitive apprenticeship. The investigators used a geological gaming simulation, the Profile
Game, to test various types of guidance structures (i.e., strategy modeling), ranging from no
guidance to illustrations (i.e., pictorial aids) to verbal descriptions to pictorial and verbal aids
combined. The Profile Game is based on the premise, “Suppose you were visiting a planet and
you wanted to determine which geological feature is present on a certain portion of the planet’s
surface” (Mayer et al., p. 171). While exploring, you cannot directly see the features, so you
must interpret data indirectly, through probing procedures. The experimenters focused on the
amount and type of guidance needed within the highly spatial simulation.
Navigation Maps and Problem Solving: revised 5/18/05
64
Though a series of experiments, Mayer et al. (2002) found that pictorial scaffolding, as
opposed the verbal scaffolding, is needed to enhance performance in a visual-spatial task. In the
final experiments, participants were divided into verbal scaffolding, pictorial scaffolding, both,
and no scaffolding. Participants who received pictorial scaffolding solved significantly more
problems than those who did not receive pictorial scaffolding. Students who received strategic
scaffolding did not solve significantly more problems than students who did not receive strategic
scaffolding. While high-spatial participants performed significantly better than low-spatial
students, adding pictorial scaffolding to the learning materials helped both low- and high-spatial
students learn to use the Profile Game. Students in the pictorial-scaffolding group correctly
solved more transfer problems than students in the control group. However, pictorial scaffolding
did not significantly affect the solution time (speed) of either low- or high-spatial participants.
Overall, adding pictorial scaffolding to the learning materials lead to improved performance on a
transfer task for both high- and low-spatial students in the Profile Game (Mayer et al., 2002).
Contiguity effect
The contiguity effect addresses the cognitive load imposed when multiple sources of
information are separated (Mayer & Moreno, 2003; Mayer, Moreno, Boire, & Vagge, 1999;
Mayer & Sims, 1994; Moreno & Mayer, 1999). There are two forms of the contiguity effect:
spatial contiguity and temporal contiguity. Temporal contiguity occurs when one piece of
information is presented prior to other pieces of information (Mayer & Moreno, 2003; Mayer et
al., 1999; Moreno & Mayer, 1999). Spatial contiguity occurs when modalities are physically
separated (Mayer & Moreno, 2003). This study is concerned with spatial contiguity, since the
printed navigation maps will be spatially separated from the 3-D video game environment.
Contiguity results in split-attention (Moreno & Mayer, 1999).
Navigation Maps and Problem Solving: revised 5/18/05
65
Split Attention Effect
When dealing with two or more related sources of information (e.g., text and diagrams),
it’s often necessary to integrate mentally corresponding representations (verbal and pictorial) to
construct a relevant schema to achieve understanding. When different sources of information are
separated in space or time, this process of integration may place an unnecessary strain on limited
working memory resources (Atkinson et al., 2000; Mayer & Moreno, 1998). In this study, the
printed navigation maps are spatially separated from the 3-D video game environment, thereby
inducing the split-attention effect.
Summary of scaffolding
Depending upon the researcher, scaffolding has several meanings: the process of forming
and building upon a schema (Chalmers, 2003); all devices or strategies that support learning (van
Merrionboer et al., 2003), the process of diminishing support as learners acquire expertise (van
Merrionboer et al., 2002); and the process of training a student on core concepts and then
gradually expanding the training. What these four definitions have in common is that scaffolding
is related to providing support during learning.
Clark (2001) described instructional methods as external representations the external
processes of selecting, organizing, and integrating. Instructional methods also provide learning
goals (Alessi, 2000; Clark, 2001), monitoring (Clark, 2001), feedback (Alessi, 2000; Clark,
2001; Leemkuil et al., 2003), selection (Alessi, 2000; Clark, 2001), hints and prompts (Alessi,
2000), and various advance organizers (Jones et al., 1995). Each of these components either
reflects a form of scaffolding or reflects a need for scaffolding.
Mayer et al (2002) argued that a major instructional issue in learning by doing within
simulated environments concerns the proper type of guidance (i.e., scaffolding). One form of
Navigation Maps and Problem Solving: revised 5/18/05
66
scaffolding is graphical scaffolding. According to Allen (1997), selection of appropriate text and
graphics can aid the development of mental models, and Jones et al. (1997) commented that
visual cues such as maps help learners conceptualize the organization of the information in a
program (i.e., the learning space). A number of studies have supported the use of maps as visual
aids and organizers (Benbasat & Todd, 1993: Chou & Lin, 1998; Ruddle et al, 1999, Chou et al.,
2000; Farrell & Moore, 2000-2001)
Graphic organizers organize information in a graphic format, which act as spatial displays
of information that can also act as study aids (Chalmers, 2003), and Jones et al. (1995) argued
that interactive designers should provide users with visual or verbal cues to help them navigate
through unfamiliar territory. Cobb argued that cognitive load can be distributed to external media
(Cobb, 1997). In virtual environments, navigation maps help to navigate and to orient the user,
and facilities an easier learning experience (Yair et al, 2001). While navigation maps provide
external representations of information needed to complete tasks and to learn, thereby reducing
or distributing cognitive load (Cobb, 1997), they also have the potential to add load, ultimately
counteracting their positive effects. The contiguity effect addresses the cognitive load imposed
when multiple sources of information are separated (Mayer et al., 1999). Spatial contiguity
occurs when modalities are physically separated (Mayer & Moreno, 2003), such as a video game
screen and a printed navigation map. The split attention effect, which is related to the contiguity
effect, occurs when dealing with two or more related sources of information (e.g., information on
a screen and information on a printed navigation map). When different sources of information
are separated in space or time, this process of integration may place an unnecessary strain on
limited working memory resources (Atkinson et al., 2000). Therefore, when working with
Navigation Maps and Problem Solving: revised 5/18/05
67
navigation maps, careful consideration must be included for the potential adverse effects of
having the navigation map separated from the learning environment.
Summary of the Literature Review
Cognitive Load Theory is based on the assumptions of a limited working memory with
separate channels for auditory and visual/spatial stimuli, and a virtually unlimited capacity longterm memory that stores schemas of varying complexity and level of automation (Brunken et al.,
2003). According to Paas et al. (2003), cognitive load refers to the amount of load placed on
working memory. Cognitive load can be reduced through effective use of the auditory and
visual/spatial channels, as well as schemas stored in long-term memory.
Meaningful learning is defined as deep understanding of the material and is reflected in
the ability to apply what was taught to new situations; i.e., problem solving transfer. (Mayer &
Moreno, 2003). Meaningful learning requires effective metacognitive skills: the management of
cognitive processes (Jones, Farquhar, & Surry, 1995), including selecting relevant information,
organizing connections among the pieces of information, and integrating (i.e., building) external
connections between incoming information and prior knowledge that exists in long-term memory
(Harp & Mayer, 1998). Related to meaningful learning is problem solving, “cognitive processing
directed at transforming a given situation into a desired situation when no obvious methods of
solution is available to the problem solver” (Baker & Mayer, 1999, p. 272). O’Neil’s Problem
Solving model (O’Neil, 1999) defines three core constructs of problem solving: content
understanding, problem solving strategies, and self-regulation. Each of these components is
further defined with subcomponents.
There are three types of cognitive load that can be defined in relationship to a learning
or problem solving task: intrinsic cognitive load (load from the actual mental processes involved
Navigation Maps and Problem Solving: revised 5/18/05
68
in creating schema), germane cognitive load (load from the instructional processes that deliver
the to-be-learned content), and extraneous cognitive load (all other load). An important goal of
instructional design is to balance intrinsic, germane, and extraneous cognitive loads to support
learning outcomes (Brunen et al., 2003).
Learner control, which is inherent in interactive computer-based media, allows for
control of pacing and sequencing (Barab, Young, & Wang, 1999). It also can induce cognitive
overload in the form of disorientation; loss of place (Chalmers, 2003), and is a potential source
for extraneous cognitive load. These issues may be the cause of mixed reviews of learner control
(Bernard, et al, 2003; Niemiec, Sikorski, & Wallberg, 1996; Steinberg, 1989), particularly in
relationship to novices versus experts (Clark, 2003c).
Computer-based educational games fall into three categories: games, simulations, and
simulation games. Games are linear, consist of rules, can contain imaginative contexts, are
primarily linear, and include goals as well as competition, either against other players or against
a computer (Gredler, 1996). Simulations display the dynamic relationship among variables
which change over time and reflect authentic causal processes and have a goal of discovering
causal relationships through manipulation of independent variables. Simulation games are a
blend of games and simulations (Gredler, 1996).
Clark (2003d) argued that mental effort is affected by motivation, and beginning with
the work of Malone (1981), a number of constructs have been described as providing the
motivational aspects of games: fantasy, control and manipulation, challenge and complexity,
curiosity, competition, feedback, and fun. Fantasy evokes “mental images of physical or social
situations that do not exist” (Malone & Lepper, 1987, p. 250). Control and manipulation promote
intrinsic motivation, because learners are given a sense of control over their choices and actions
Navigation Maps and Problem Solving: revised 5/18/05
69
(deCharms, 1986, Deci, 1975). Challenge embodies the idea that intrinsic motivation occurs
when there is a match between a task and the learner’s skills (Bandura, 1977, Csikszentmihalyi,
1975; Harter, 1978). For challenge to be effective, the the task should be neither too hard nor too
easy, otherwise the learner would lose interest (Clark, 1999; Malone & Lepper, 1987). Curiosity
is related to challenge and arises from situations in which there is complexity, incongruity, and
discrepancy (Davis & Wiedenbeck, 2001).
Studies on competition with games and simulations have resulted in mixed findings,
due to individual learner preferences, as well as the types of reward structures connected to the
competition (e.g., Porter, Bird, & Wunder, 1990-1991; Yu, 2001). Another motivational factor in
games, feedback, allows learners to quickly evaluate their progress and can take many forms,
such as textual, visual, and aural (Rieber, 1996). Ricci et al. (1996) argued that feedback can
produce significant differences in learner attitudes, resulting in increased attention to a learning
environment. However, Clark (2003a) commented that feedback must be focused on clear
performance goals and current performance.
The last category contributing to motivation, fun, is possibly an erroneous category.
Little empirical evidence exists for the construct. However, evidence does support the related
constructs of play, engagement, and flow. Play is entertainment without fear of present of future
consequences (Resnick & Sherer, 1994). Csikszentmihalyi (1975; 1990) defines flow as an
optimal experience in which a person is so involved in an activity that nothing else seems to
matter. According to Davis and Wiedenbeck (2001), engagement is the feeling of working
directly on the objects of interest in a world, and Garris et al. (2002) argued that engagement can
help to enhance learning and accomplish instructional objectives.
Navigation Maps and Problem Solving: revised 5/18/05
70
While numerous studies have cited the learning benefits of games and simulations
(e.g., Adams, 1998; Baker et al., 1997; Betz, 1995-1996; Khoo & Koh, 1998), others have found
mixed, negative, or null outcomes from games and simulations, specifically in relationship to the
of enjoyment of a game to learning from the game (e.g., Brougere, 1999; Dekkers & Donatti,
1981; Druckman, 1995). There appears to be consensus among a large number of researcher with
regards to the negative, mixed, or null findings, suggesting that the cause might be a lack of
sound instructional design embedded in the games (de Jong & van Joolingen, 1998; Garris et al.,
2002; Gredler, 1996; Lee, 1999; Leemkuil et al., 2003; Thiagarajan, 1998; Wolfe, 1997). Among
the various instructional strategies, reflection and debriefing have been cited as critical to
learning with games and simulations.
An important component of research on the effectiveness of educational games and
simulations is the measurement and assessment of performance outcomes from the various
instructional strategies embedded into the games or simulations, such as problem solving tasks.
Problem solving is cognitive processing directed at achieving a goal when no solution method is
obvious to the problem solver (Baker & Mayer, 1999). O’Neil’s Problem Solving model (O’Neil,
1999), includes the components: content understanding; solving strategies—domain-independent
(-general) and domain-dependent (-specific); and self-regulation which is comprised of
metacognition and motivation. Metacognition is further composed of self-checking/monitoring
and planning, and motivation is comprised of effort and self-efficacy. Knowledge maps are
reliable and efficient for the measurement of the content understanding portion of O’Neil’s
Problem Solving model, and CRESST has developed a simulated Internet Web space to evaluate
problem solving strategies such as information searching strategies and feedback inquiring
strategies.
Navigation Maps and Problem Solving: revised 5/18/05
71
Problem-solving can play a large amount of cognitive load on working memory.
Instructional strategies have been recommended to help control or reduce that load. One such
strategy is scaffolding. While there are a number of definitions of scaffolding (e.g., Chalmers,
2003; van Merrionboer et al., 2002; van Merrionboer et al., 2003), what they all have in common
is that scaffolding is an instructional method that provides support during learning. Clark (2001)
described instructional methods as external representations the external processes of selecting,
organizing, and integrating. They provide learning goals, monitoring, feedback, selection, hints,
prompts, and various advance organizers (Alessi, 2000; Clark, 2001; Jones et al., 1995; Leemkuil
et al., 2003). Each of these components either reflects a form of scaffolding or reflects a need for
scaffolding
One form of scaffolding is graphical scaffolding. A number of studies have reported
the benefits of maps, which is a type of graphical scaffolding (Benbasat & Todd, 1993: Chou &
Lin, 1998; Ruddle et al, 1999, Chou et al., 2000; Farrell & Moore, 2000-2001). In virtual
environments, navigation maps help to navigate and to orient the user, and facilities an easier
learning experience (Yair et al, 2001). While navigation maps can reduce or distribute cognitive
load (Cobb, 1997), they also have the potential to add load, ultimately counteracting their
positive effects. The spatial contiguity effect addresses the cognitive load imposed when multiple
sources of information are separated (Mayer & Moreno, 2003) and the split attention effect,
which is related to the contiguity effect, occurs when dealing with two or more related sources of
information (Atkinson et al., 2000). Navigation maps can provide value cognitive support for
navigating virtual environments, such as computer-based video games. This can be particularly
useful when using the gaming environment to accomplish a complex problem-solving task.
Navigation Maps and Problem Solving: revised 5/18/05
72
CHAPTER III: METHODOLOGY
Research Design
University of Southern California Human Subjects approval was requested on June 17,
2004. Revisions were requested on July 15 for the recruitment flyer and the Informed Consent
Form. Changes to these two forms were made and resubmitted on July 22, 2004. On July 26,
2005, the USC Institutional Review Board (IRB) approved all forms, allowing subjects to be
contacted and the experimental phase of study to begin.
The design of the study is true experimental, with randomized selection and assignment
of participants, and involves intermediate-, and post-tests for one treatment group and one
control group. Subjects were randomly assigned to either the treatment or the control group.
Group sessions involved only one group type: either all treatment participants or all control
participants. Due to limited availability of computers, session size was limited to a maximum of
three participants. At the end of the 90 minute session, participants were debriefed and allowed
to continue playing on their own for up to 30 additional minutes (to examine continued
motivation).
Research Questions and Hypotheses
Research Question 1: Will the problem solving performance of participants who use a
navigation map in a 3-D, occluded, computer-based video game (i.e., SafeCracker®) be better
than the problem solving performance of those who do not use the map (the control group)?
Hypothesis 1: Navigation maps will produce a significant increase in content
understanding compared to the control group.
Hypothesis 2: Navigation maps will produce a significant increase in problem solving
strategy retention compared to the control group.
Navigation Maps and Problem Solving: revised 5/18/05
73
Hypothesis 3: Navigation maps will produce a significant increase in problem solving
strategy transfer compared to the control group.
Hypothesis 4: There will be no significant difference in self-regulation between the
navigation map group and the control group. However, it is expected that higher levels of selfregulation will be associated with better performance.
Research Question 2: Will the continued motivation of participants who use a
navigation map in a 3-D, occluded, computer-based video game (i.e., SafeCracker®) be greater
than the continued motivation of those who do not use the map (the control group)?
Hypothesis 5: Navigation maps will produce a significantly greater amount of optional
continued game play compared to the control group.
Sample
Two samples were selected solicited for this study. The first sample consisted of two
participants for the pilot study conducted September 28 and 29, 2004. The purpose of the study
was to review the procedures and instruments that were to be utilized in the main study. This
sample was a convenience sample and consisted of one female approximately 49 years 4 months
of age and a male approximately 32 years and 8 months of age. The female was a graduate of the
University of Southern California, Los Angeles, California. The male was a graduate of
California State University, Northridge, California. Both participants had a reasonable level of
computer proficiency and neither had prior experience with the game SafeCracker©.
Between November 11, 2004 and March 21, 2005, sixty-six English-speaking adults,
ranging in age from 19 years and 4 months to 31 years and 11 months, participated in the main
study. The average participant age was a few days less than 23 years. All the participants for the
Navigation Maps and Problem Solving: revised 5/18/05
74
main study were undergraduate students, graduates, or graduate students of the University of
Southern California, Los Angeles. California.
Solicitation of participants. Participation was solicited through several methods. The
primary method was a small standard paper size (8 and a half by 11 inch) flyer (Appendix A)
posted in various locations within five of the university’s schools; the Marshall School of
Business, the Viterbi School of Engineering, the Annenberg School for Communication, the
School of Cinema, and the Rossier School of Education. These schools were chosen for a
number of reasons, including; how many locations the allowed flyers to be posted at; ease of, or
ability to get, approval to post flyers; a belief that their students might be interested in
participating in a video game study.
Flyers (Appendix A) were also sent via email attachment to two of the university’s
student organizations that the researcher believed would contain students potentially interested in
this type of study. The organizations were a student video game development group and a
student television and film special effects group. The researcher is the faculty advisor to the
video game group. Flyers were also posted around campus at locations approved for display of
announcements. These locations included student congregation areas, major outdoor pathways,
and parking structure stairwells. The flyer included a statement that participants would be paid
$15 for approximately 90 minutes of participation, and participants must have no prior
experience playing the personal computer (PC)-based video game SafeCracker® (Daydream
Interactive, 1995). Email contact information was provided on the flyer.
Randomized assignment. As requests for participation arrived, each participant was
randomly assigned to either the treatment or control group. Participants were entered into a
Microsoft Excel 2002 spreadsheet for tracking purposes, in the order in which their emails were
Navigation Maps and Problem Solving: revised 5/18/05
75
received. Randomized assignment was accomplished using a random number generator within a
Microsoft Excel 2002 spreadsheet. The spreadsheet was used for tracking and other logistical
issues related to the study. When the participant’s last name was entered and either the enter or
tab key was pressed on the computer, The generator would display a number between
0.000000000 and 1.000000000, in increments of .000000001. If the number was from
0.000000000 to 0.500000000, the participant was assigned to the Control group. If the number
was from 0.510000000 and 1.000000000, the participant was assigned to the Treatment group.
Various study times were selected for each group and students were sent those times relevant to
their group. Participants responded by listing one or more times during which they could
participate. From the responses, the researcher scheduled participants to best fill each available
time slot.
Number of particpants whose data were analyzed. A total of 71 students participated in
the study, with 69 completing the study; Thirty-five received the treatment and thirty-four were
in the control group. Those in the treatment group received a navigation map, which was an
overhead view of the game’s playing environment; a mansion (figure 4). Those in the treatment
group also received instruction on how to read the map and use the map for planning and
navigation. Those in the control group did not receive the navigation map and were only given
brief instruction on how to navigate without a map.
Of the 71 students that participated in the study, 69 completed the study, but the data
from only 66 participants are included in this study. In one session involving two participants,
the researcher inadvertently had the participants overwrite a file with some of their prior data,
making the comparison between intermediate data and post data impossible. Those two
participants were not included in the data analysis. Three other participants did not complete the
Navigation Maps and Problem Solving: revised 5/18/05
76
study, due to computer errors. Near the end of the experimental phase of the study, two of the
computer began to exhibit problems. One computer began to freeze (quit accepting or responding
to user input). The other computer began to intermittently display an error during the second
round of game play during a session. In most cases, turning off the computers before various
phases of the study alleviated or prevented problems. However, near the end of the data
collection phase of the study, the computer that had intermittently been freezing began to
regularly freeze. Two participants had to end the study early and not enough data was collected
both either participant to be analyzable. From that point onward, that computer was not used in
the study, limiting the number of participants for any session to two. For the other computer,
restarting the computer at various phases worked well for all but one participant. This participant
had to leave the study early and not enough data was collected for analysis. It should also be
noted that two participants had to leave very early in the study due to computer problems.
However, in both cases, the subjects had only been shown how to use one software package (the
Knowledge Mapping software). It was determined to be acceptable to have those two subjects
return to complete the study at a later date without compromising the validity of their data. They
did return and complete the study.
Hardware
The pilot study took place in the home office of the researcher. The computer utilized for
the study was a 450 MHz (megahertz) desktop computer made by Tiger Direct
(http://www.tigerdirect.com) with 64 MB (megabytes) of RAM (random-access memory), a
standard computer keyboard, a 3-button mouse, and a 21” CRT (cathode-ray tube) monitor.
The main study took place in the campus office of the researcher. A table was set up for
two computers placed side by side. One computer was a Pentium 200, NeTPower Symmetra
Navigation Maps and Problem Solving: revised 5/18/05
77
computer with 128 MB of RAM that originally ran the Windows NT operating system, but was
installed with the Windows 98 operating system for the study. The computer configuration
included a standard computer keyboard, a 3-button mouse, and a 17” CRT monitor. Another
computer was a Sony PCG-F520, Pentium III laptop computer with 192MB RAM, running the
Windows 98 operating system. The laptop’s keyboard was used, but an external USB (universal
serial-bus) 2-button mouse was added. A 14” CRT monitor was attached to this computer,
because the laptop’s built-in LCD (liquid crystal display) monitor was not very good; the
displayed images were very dark and had low contrast, and the screen exhibited a lot of
reflection and glare, making visibility difficult. This computer was placed on the table next to the
NeTPower Symmetra. The Symmetra’s power case was placed on the desk between the two
computers, to reduce the visibility by participants of each other’s monitor. The third computer
was a Dell Latitude D500, Pentium M laptop computer with 256MB RAM, and running the
Windows 98 operating system. Similar to the other laptop, this laptop’s keyboard was used, but a
serial bus 2-button mouse was added. This computer was placed on a lateral file cabinet. A
monitor was not attached to this laptop, because its built-in 12” LCD screen produced a
satisfactory picture. The three computers were placed so that participants could not readily see
what other participants were doing and participants had enough room to use the mouse and to
write on paper. The primary mode of computer input and interaction during the study was via the
mouse. The only time the keyboard was used was for entering a file name when saving various
types of data. Two files were saved during one phase of the study and two files were saved
during a second phase of the study, for a total of four files.
Instruments
Navigation Maps and Problem Solving: revised 5/18/05
78
A number of instruments will be included in the study: a demographic/game preference
form, a task completion form, a self-regulation questionnaire, the computer-based video game
SafeCracker®, a navigation map of the game’s environment, a problem-solving retention and
transfer questionnaire; and knowledge mapping software.
Demographic, game play, and game preference questionnaire
At the start of the experiment, a questionnaire (Appendix B) was administered to elicit
gender, age, amount of weekly video game play, and preferred game types. For gender,
participants marked either the male or female box. For age, participants entered both the number
of years and the number of months. For amount of weekly video game play, participants checked
one of four boxes: none, 1 to 2 hours, 3 to 6 hours, and greater than 6 hours. The game types
section listed Puzzle games, RTS games, FPS games, Strategy games, Role Playing games,
Arcade games, PC games, and Console games. The first five items in this section are game
genres. The last two items are game platforms. The sixth item, Arcade game, can be either a
game genre or platform. For each of the preferred game types, participants entered a number
from 1 to 5, with 1 indicating low interest and 5 indicating high interest. Participants were
prompted to enter a zero if they did not play that game type, were not sure of their response, or
did not know the particular type of game type or what the initials meant. It was determined by
the researcher that those who played RTS or FPS games, in particular, would know what those
terms meant, since the terms were commonly used by players of those particular game types;
RTS stands for Real-Time Strategy and FPS stands for First-Person Shooter (also known as first
person perspective). If a participant asked what a term meant, he or she was told to enter a zero.
The last two game types are gaming platforms. PC games, which stand for Personal
Computer games, refer to games played on personal, or home, computers (PCs), such as an
Navigation Maps and Problem Solving: revised 5/18/05
79
Apple computer or Windows-based computer. Console games are those games played on gaming
consoles, such as PlayStation, X-Box, or Nintendo game consoles.
Arcade games refers to both a genre and a platform. As a genre, arcade games are short,
and often rapid reaction, games with short playing durations, and only one or two goals. As a
game platform, arcade games historically refers to games played on large stand-alone gaming
consoles like those found in public arcades. Today, however, arcade games are also available on
home computers (PCs).
The divisions for amount of weekly game play included in the questionnaire (none, 1 to 2
hours, 3 to 6 hours, and greater than 6 hours) were based on a study conducted in 1996 by the
Media Analysis Laboratory, Simon Fraser University, Burnaby, British Columbia, Canada. The
study surveyed 647 children ranging in age from 11 to 18, with 80% between the ages of 13 and
15. Six hundred forty six subjects completed the survey (351 male and 295 female). Most
children surveyed in this study played video games between 1 and 6 hours per week. Based on
the findings of this study, which indicated that most children surveyed played between 1 and 6
hours per week, the four divisions used in this study were created. Information on the British
Columbia study can be found at http://www.mediaawareness.ca/english/resources/research_
documents/studies/video_games/vgc_vg_and_television.cfm
Task completion form
Immediately before the start of each game (the game was played twice during the study),
participants were handed a Task Completion form (Appendix C). This form listed the names of
the rooms that were involved in a particular game and the safes that could be found in each room.
Players were told to mark off (check the boxes for) each safe they opened and to be sure to mark
them off as soon as a safe was opened, so as not to forget which safes were opened during a
Navigation Maps and Problem Solving: revised 5/18/05
80
game. At the end of each game, players were prompted to check the form to ensure all opened
safes were marked off.
Self-Regulation Questionnaire
A trait self-regulation questionnaire (Appendix D) designed by O’Neil and Herl (1998)
was administered to assess each participant’s degree of self-regulation, which is one of the three
components of problem-solving ability as defined by O’Neil (1999). Reliability of the instrument
ranges from .89 to .94, as reported by O’Neil & Herl (1998). The 32 items on the questionnaire
are composed of eight items each of the four self-regulation factors in the O’Neil (1999) Problem
Solving model (see figure 1): planning, self-checking/monitoring, self-efficacy, and effort. For
example, item 1 (Appendix D) “I determine how to solve a task before I begin.” is designed to
assess a participant’s planning ability; and item 2 “I check how well I am doing when I solve a
task” is to evaluate a participant’s self-efficacy. The response format for each item is a Likerttype scale with four possible responses; almost never, sometimes, often, and almost always. The
self-regulation form was administered in printed format, with participants using either a pen or
pencil to enter responses (a number from 1 to 4) for each question. Responses were later entered
into a Microsoft Excel 2002 spreadsheet and totals for each for the four self-regulation factors
were generated using Excel’s SUM function.
SafeCracker
The non-violent, PC-based video game SafeCracker® (Daydream Interactive, 1995) was
selected for this study, as a result of a feasibility study by Wainess and O’Neil (2003). The
purpose of the feasibility study was to recommend a video game for use as a platform for
research on cognitive and affective components of problem solving, based on the O’Neil (1999)
Navigation Maps and Problem Solving: revised 5/18/05
81
Problem Solving Model (see figure 1). The following requirements were established for a game
to be considered as a potential platform.
Required characteristics:
*
Adult focused (college age and above)
*
Appeal to both genders and with broad audience appeal
*
Single user play
*
Suitable for problem solving research
*
Game to support practice and include problem solving feedback
*
Interface, controls, and navigation can be adequately mastered in half an hour or less
*
Can be played multiple times (or multiple stages) in one hour
*
Compatible with X-Box, Playstation 2, or PC

Expert support available for analysis of play/problem solving
The following additional criteria are preferred, but not required.
*
Non-violent game
*
User controlled pacing of game action
*
Multi-user (multiplayer) capable
*
Suitable for study of retention and transfer
*
Provides multiple conditions or can be programmed with varying conditions
*
Ability to track actions (e.g. time, clicks, choices)
The feasibility study by Wainess and O’Neil (2003) involved four phases: 1) selection of
relevant game types based on simple search criteria; 2) in-depth analysis of the most relevant
game selections; 3) removal of similar games by selecting the most appropriate game among the
equivalent games; 4) play testing of each remaining game to determine the final selection. Phase
Navigation Maps and Problem Solving: revised 5/18/05
82
1 began with selection of 525 potential games and ended with the selection of 12 possible games.
Phase 2 ended with the selection of four possible games, and phase 3 ended with the selection of
three possible games: SafeCracker for Windows®, Myst III®: Exile for Windows/PS2/Xbox©,
and Jewels of the Oracle for Windows©. Phase 4 resulted in the selection of SafeCracker®
(Wainess & O’Neil, 2003).
According to Wainess and O’Neil (2003), the primary factor for selecting
SafeCracker® over the other two possible games selected during phase 3 was time constraints.
During the feasibility study, it had been decided that participants should not be required to spend
more than one and a half hours in any study for which the game would be used. In addition, it
was desirable to include multiple iterations of gameplay within that time period. In Phase 4,
Myst III®: Exile© and Jewels of the Oracle© were both eliminated due to the number of hours
required for problem solving research. The games are designed with large environments,
numerous buildings and other structures, and myriad, complex clues that could require multiple
hours of play time to accomplish a set of problem solving tasks. With SafeCracker, players
would be able to learn the controls and interface and enter the main game environment (a
mansion) in approximately 15 minutes. Because only two or three of the mansion’s
approximately 50 rooms are needed for an effective study, problem solving events could occur in
10 or 20 minutes, allowing for multiple problem-solving tasks using different room
combinations.
Navigation map
Gameplay in SafeCracker takes place in a two story mansion. For the purposes of this
study, only a small number of rooms on the first floor were utilized. A navigation map, in the
form of a topological floor plan of the first floor, was downloaded from
Navigation Maps and Problem Solving: revised 5/18/05
83
http://www.gameboomers.com/wtcheats/pcSs/Safecracker.htm. The navigation map was
subsequently modified using Adobe® Photoshop® 6.5, to alter the view of the navigation map
from one-point perspective to a flat 2-D image, to remove unnecessary artifacts, to remove room
numbers for each room, to add a name to each room in accordance with names displayed on the
game’s interface, and to add a compass symbol to the top right side of the map. Figure 4 shows
the final, modified version of the map.
Figure 4: Navigation Map
Knowledge Map
In this study, participants were asked to create a knowledge map in a computer-based
environment to evaluate their content understanding after playing SafeCracker. According to
Plotnick (1997), a knowledge map, referred to as a concept map by Plotnick, is a “graphical
representation where nodes (points or vertices) represent concepts, and links (arcs or lines)
represent the relationships between concepts” (p. 81). He further commented that the concepts
Navigation Maps and Problem Solving: revised 5/18/05
84
and links are labeled on the map and that the links could be unidirectional, bi-directional, or nondirectional.
During the study, participants played SafeCracker twice and completed a knowledge map
after each game session. The computerized knowledge map used in this study had been
successfully applied to other studies (e.g., Chuang, 2003; Chung et al., 1999; Hsieh, 2001;
Schacter et al., 1999). Appendix F lists the knowledge map specification used in this study
(adapted from Chen, in preparation)
Content understanding measure. Content understanding measures were computed by
comparing the semantic content score of a participant’s knowledge map to the average semantic
score of two subject matter experts. The experts for this study are Richard Wainess and Hsin-Hu
(Claire) Chen. Appendix E shows the expert knowledge map developed for this study. It is based
on the general concepts and propositions relevant to problem solving in the game SafeCracker,
not the concepts and propositions specific to a room or a safe. Figure 5 shows a sample of a
knowledge map or portion of a knowledge map that might be created during this study.
Figure 5: Knowledge map for the game SafeCracker®
contains
room
key
results from
results in
crack
map
causes
clue
Navigation Maps and Problem Solving: revised 5/18/05
85
Scoring the knowledge map. The following describes how the knowledge map will be
scored. The semantic score is calculated based on the semantic propositions—two concepts
connected by one link in the experts’ knowledge map. Every proposition in a participant’s
knowledge map is compared against each proposition in the experts’ map. A match would be
scored as one point. The average score of the two experts would be the semantic score for the
student map. For example, as seen in Table 1, if a participant made a proposition such as “room
contains key,” this proposition would then be compared against the experts’ propositions.
Table 1: An Example of Scoring Map
CONCEPT1
LINKS
CONCEPT 2
EXPERT 1
EXPERT 2
AVERAGE
Room
contains
key
1
1
1
Crack
results
Key
1
0
0.5
Crack
0
1
0.5
from
Clue
causes
Total
2.00
The participant can only receive one of two scores: the average score of the two experts
or a score of zero. In the case of “room contains key,” a score of one (the average score of the
two experts) would mean the participant’s proposition was the same as the proposition of at least
one expert. A score of zero would mean the proposition does not match either expert’s
proposition. If the participant then created the link “room contains key” he or she would receive
an addition score of one point to reflect the average score of the two experts, for a total of two
points (one point for the semantic link “room” and one point for the proposition “room contains
key”). Selecting “crack” would receive one point (for matching at least one expert). Creating the
link (the proposition) “crack results in key” would garner an additional half-point (the average
score of the two experts), for a total of one and one-half points.
Navigation Maps and Problem Solving: revised 5/18/05
86
Domain-specific Problem-Solving Strategies Measure
In this study, a problem solving instrument successfully employed by Richard Mayer and
Roxanna Moreno (see for example, Mayer, 2001; Mayer et al, 2003; Mayer & Moreno, 1998;
Mayer et al., 2003; Moreno & Mayer, 2000, 2004) was modified to measure the domain specific
problem-solving strategies of the game SafeCracker. In one study, Mayer and Moreno (2003)
measured retention by having participants respond to an opened ended domain-specific question
“Please write down an explanation of how lightening works.” Acceptable answers, referred to as
idea units, were defined by the researchers. Participants’ responses were then compared to those
idea units and considered valued if the content matched, regardless of exact wording. Idea units
defined by the researchers included “air rises,” “water condenses,” “water and crystals fall,” and
“wind is dragged downward.” Each response a participant wrote that matched an idea unit
received one point. Retention scores were determined by totaling the number of matches, with a
higher score indicating higher retention.
In the same study by Mayer and Moreno (2003), participants were given the transfer
question “Suppose you switch on an electric motor, but nothing happens.” What could have gone
wrong?” For this question, the researchers generated a list of acceptable idea units (answers)
such as “the wire loop is stuck,” or “the wire is severed or disconnected from the battery” and
participants’ responses were compared to these idea units. As with the retention responses, one
point was given to each response that matched one of the researchers’ idea units, regardless of
wording. A participant’s retention score was the sum of matches.
As will be discussed later, a modified version of the Mayer and Moreno (2003) procedure
was utilized for this study. Instead of giving one point for each idea unit matched, idea units
were given different values, based on Anderson et al’s (2001) Taxonomy Table for learning (see
Navigation Maps and Problem Solving: revised 5/18/05
87
figure 2). The table includes two dimensions of knowledge, creating a matrix for knowledge with
more meaningful representations or knowledge going from left to right and top to both.
Therefore, the top left cell of the matrix represents shallow or rote knowledge and the bottom
right cell represents deep or meaningful knowledge.
The problem-solving strategy questions designed for this dissertation research were one
retention question and one transfer question relevant to problem-solving using SafeCracker, the
selected puzzle-solving game. Participants were given four sheets of paper clipped together. Near
the top of page one was the retention question “List the ways you found rooms and opened
safes.” Following the retention question were forty-two double-spaced and number lines
spanning to the end of page two. Near the top of page three was the transfer question “List some
ways to improve the design of the game play for opening safes.” Following the transfer question
were forty-two double-spaced and numbered lines spanning to the end of page four. The two
questions, retention and transfer, were designed to elicit responses that would indicate an
understanding of not only game play related to specific safes (i.e., the retention question), but for
overall understanding the game and game play (i.e., the transfer question).
Retention question:
– Write an explanation of how you solve the puzzles in the rooms:
Transfer questions:
– List some ways to improve the fun or challenge of the game:
The idea units used in this study for both retention and transfer responses, were created
by Chen (in preparation) for an earlier study utilizing SafeCracker. For her study, the idea units
for the retention and transfer questions were created by the five experts. Independently, the
experts created idea units/propositions for the retention and transfer questions. Then all answers
Navigation Maps and Problem Solving: revised 5/18/05
88
were pooled into two lists, one for the retention question and the other for the transfer question.
Next, through discussion of the two lists, the experts analyzed and selected the final idea
units/propositions that would represent valid problem-solving strategy responses to the retention
and transfer questions. The result was 29 idea units for the retention question and 22 idea units
for transfer question as listed in Table 2 and Table 3 respectively.
Table 2
Idea Units of the Retention Question
1
Recognize/compare room features.
2
Recognize/compare safe features.
3
Search for clues/hints
4
Search for keys/ tools.
5
Remember clues
6
Identify objects
7
Identify clues
8
Look for clue in book/catalog/ brochure
9
Looking/walking around
10 Get more familiar with different kinds of safe
11 Get implicit feedback
12 Get familiar with surrounding
13 Guessing/ trail and error
14 Use elimination method
15 Use clues/hints
16 Follow compass and floor plan/map.
Navigation Maps and Problem Solving: revised 5/18/05
17 Figure direction of the target rooms.
18 Figure out the procedure/process/pattern/system/sequence
19 Use tools
20 Jot down notes.
21 Evaluate if a previous strategy is effective
22 Analyze what to do next
23 Analyze what the problem/difficulty is.
24 Plan strategies of safecracking
25 Change a strategy
26 Combine clues and hints
27 Apply subject knowledge such as math and science
28 Use common sense/daily-life knowledge
29 Logical reasoning
Table 3
Idea Units of the Transfer Question
1
Add rules such as “can not go back” or "no note-taking"
2
Add characters to disturb/confuse or help
3
Add more on-screen help
4
Add confusing objects or information/clues
5
Increase/decrease the number or variety of rooms.
6
Increase/decrease the number or variety of puzzles/safes.
7
Increase/decrease the number or variety of tools/objects
89
Navigation Maps and Problem Solving: revised 5/18/05
8
Increase/decrease the number or variety of clues/hints
9
Require more/less clues for a puzzle/safe/room
90
10 Require more/less tools for a puzzle/safe/room
11 Require more/less complicated process/procedure/sequence
12 Make safes/puzzles more/less confusing/complicated.
13 Make rooms more/less confusing/complicated.
14 More connection among objects/clues/safes/rooms
15 More interaction/feedback between player and the computer/game
16 Increase audio effect such as adding nervous/tension music
17 More interesting/dramatic background story/ gist/ meaning
18 Alternatives of solutions/Multiple solutions.
19 Better extra/exterior helper/tools.
20 Change the length of playing time/ time limitation.
21 Skip/more previous instruction or no/more previous demonstration.
22 Explanation of game strategies/Strategy guidance.
Scoring of the Participants’ Problem-Solving Strategies Retention and Transfer Reponses.
According to Anderson et al. (2001), a taxonomy is a type of framework, in which the
categories lie along a continuum. The Anderson et al. Taxonomy Table classifies educational
objectives. In education, a statement of an educational objective contains a verb and a noun (e.g.,
“Learn addition”). The verb generally describes the intended cognitive process (“learn” in the
example given), while the noun generally describes the knowledge to be acquired or constructed
(“addition” in the example given). The two dimensions represented in Taxonomy Table (Figure
Navigation Maps and Problem Solving: revised 5/18/05
91
2) are cognitive process and knowledge, and Anderson et al. refer to the interrelationships
between the cognitive processes and knowledge as the Taxonomy Table.
Figure 2: Taxonomy Table
The
The cognitive process dimension
Knowledge
1.
2.
3.
4.
5.
6.
Dimension
Remember
Understand
Apply
Analyze
Evaluate
Create
A.
Factual
Knowledge
B.
Conceptual
Knowledge
C.
Procedural
Knowledge
D.
MetaCognitive
Knowledge
Categories of the knowledge dimension. There are four types of knowledge in Taxonomy
Table: Factual, Conceptual, Procedural, and Metacognitive. Anderson et al (2001) used the term
Factual Knowledge for the knowledge of discrete, isolated “bits of information” and the term
Conceptual Knowledge for more complex, organized knowledge forms (i.e., concepts, principles,
Navigation Maps and Problem Solving: revised 5/18/05
92
models, or theories). Procedural Knowledge is “knowledge of how to do something” (p. 27), and
includes knowledge of skills and algorithms, techniques and methods, and well as knowledge of
knowledge of the criteria used to determine an/or justify (p. 27).
The final knowledge form, Metacognitive Knowledge is “knowledge about cognition in
general as well as aware ness of and knowledge about one’s own cognition” (p. 27). It
encompasses strategic knowledge; knowledge about cognitive tasks, including contextual and
conditional knowledge; and self-knowledge. More specifically, it refers to knowledge about
ones own cognition and control of ones own cognition. The latter type, control, encompasses the
concept of self-regulation. These two dimensions, knowledge and control, parallel the two
dimensions in the Taxonomy Table, with knowledge representing to the knowledge dimension
and control representing the cognitive processes dimension. As such in the Taxonomy Table,
Metacognitive Knowledge refers only to the knowledge component of metacognition and not the
control or self-regulation component, which fits into the cognitive process dimension.
Each of the four knowledge types can be further analyzed into subtypes. For example,
Factual Knowledge encompasses both knowledge of terminology and knowledge of specific
details and elements. Table X lists both the four major types of knowledge and the subtypes
within each major type. For a complete explanation of the subtypes, see Anderson et al. (2001).
Table X (adapted from a table in Anderson, 2001, p. 29).
The Major Types and Subtypes of the Knowledge Dimension
Major Types and Subtypes
A.
Factual knowledge: Basic elements needed to solve problems in a discipline
a. Knowledge of terminology
b. Knowledge of specific details and elements
Navigation Maps and Problem Solving: revised 5/18/05
B.
93
Conceptual knowledge: Interrelationships among basic elements enabling them to
function together
a. Knowledge of classifications and categories
b. Knowledge of principles and generalizations
c. Knowledge of theories, models, and structures
C.
Procedural knowledge: How to do something, methods of inquiry, and criteria for
using skills, algorithms, techniques, and methods
a. Knowledge of subject-specific skills and algorithms
b. Knowledge of subject-specific techniques and methods
c. Knowledge of criteria for determining when to use appropriate procedures
D.
Metacognitive knowledge: Knowledge of cognition in general, as well as awareness
and knowledge of one’s own cognition.
a. Strategic knowledge
b. Knowledge about cognitive tasks, including appropriate contextual and
conditional knowledge
c. Self-knowledge
Categories of the cognitive process dimension. The categories of the cognitive process
dimension (Table X) range from the cognitive processes most commonly found in educational
objectives, those associated with Remember, Understand, and Apply, to those less frequently
found, Analyze, Evaluate, and Create. According to Anderson et al. (2001), Remember refers to
retrieving relevant knowledge from long-term memory. Understand is constructing the meaning
of instructional messages. Apply means carrying out or using a procedure in a given situation.
Analyze refers to breaking material into its constituent parts and determining how those parts are
Navigation Maps and Problem Solving: revised 5/18/05
related, as well as their overall structure and purpose. Evaluate consists of making judgments
based on criteria and/or standards. Create involves putting elements together to form a novel,
coherent whole or to make an original product (p. 30).
Each of the six major categories is associated with two or more specific cognitive
processes, also described as verb forms (see Table x). For example, the major category of
Remember, includes two verb forms (recognizing and recalling) while Understand includes
seven (interpreting, exemplifying, classifying, summarizing, inferring, comparing, and
explaining). See Anderson et al (2001) for a detailed description of each major category and
related verb forms.
Table X (adapted from a table in Anderson, 2001, p. 29).
The Six Categories of the Cognitive Process Dimension and Related Cognitive Processes
1. Remember: Retrieve relevant knowledge from long-term memory
1.1. Recognizing
1.2. Recalling
2. Understand: Construct meaning from instruction messages.
2.1. Interpreting
2.2. Exemplifying
2.3. Classifying
2.4. Summarizing
2.5. Inferring
2.6. Comparing
2.7. Explaining
3. Applying: Carry out or use a procedure in a given situation
94
Navigation Maps and Problem Solving: revised 5/18/05
95
3.1. Executing
3.2. Implementing
4. Analyze: Break material into constituent parts and determine how parts relate to one
another and to an overall structure or purpose
4.1. Differentiating
4.2. Organizing
4.3. Attributing
5. Evaluate: Make judgments based on criteria and standards
5.1. Checking
5.2. Critiquing
6. Create
6.1. Generating
6.2. Planning
6.3. Producing
Assigning retention and transfer idea units to a matrix cell. The Taxonomy Table (Figure
2) created by Anderson et al. (2001) defines a matrix of interrelationships between cognitive
processes (columns) and knowledge types (rows). The Taxonomy Table was designed to assist in
the assignment or categorization of educational objects. For this study, however, the table has
been adapted for coding of problem solving retention and transfer statements. In a sense, the
function of the table has been reversed. The purpose of assigning an educational objective to a
cell in the indicates what needs to be learned. In contrast, in this study, assigning a problemsolving response to the matrix indicates what has been learned.
Navigation Maps and Problem Solving: revised 5/18/05
96
The process of assigning either an educational objective or a participant response from
this study to a matrix cell begins by locating the verb and noun in the educational objective or
problem-solving retention or transfer response, respectively. The verb is examined in the context
of the six categories of the cognitive process dimension: Remember, Understand, Apply, Analyze,
Evaluate, and Create. The noun is examined in the context of the four types in the knowledge
dimension: Factual, Conceptual, Procedural, and Metacognitive. For example, “remember
clues” indicates the cognitive process of Remember and the knowledge type of Factual
Knowledge, which would place that statement in the upper left grid of the matrix (see Figure 2).
While this process appears simple, it is not that simple a task. There are two reasons for
this difficulty. First is that statements may contain more than verbs and nouns. These extras
words may be irrelevant to the classification or may provide conditions for classification (p. 33).
For example, consider the terms “search for clues” and “search for clues in books.” While one
includes the condition of using a book during the search, the addition of a book is irrelevant to
the classification. The verb search indicates the cognitive process 3.1 Apply: execute and the
noun clue category Ab in the Knowledge dimension, “Factual Knowledge: Knowledge of
specific details or elements” (see Figure 2).
Creating and even more difficult situation is the possibility that the verb may be
ambiguous in terms of intended cognitive process or the noun may be ambiguous in its intended
knowledge. For example, in the statement “find clue,” the verb find is ambiguous. Does find
indicate “Apply: execute” as did the verb search. Or does is imply something more? In order to
find something, one must execute a search (Apply), but also differentiate what was found from
other items to decide a clue was found (Analyze). Therefore, is find and indication of the
cognitive process of Apply or that of Analyze? Ultimately, classification must involve inference.
Navigation Maps and Problem Solving: revised 5/18/05
97
Because not all propositions can be easily applied to the Taxonomy Table (figure 2)
interrater reliability was used in this study for assigning retention and transfer idea units (Tables
2 and 3 respectively) to matrix cells. This researcher, along with another researcher using the
same computer-based game SafeCracker, and conducting related research on problem-solving
(see Shen, in preparation), independently assigned the 29 problem-solving retention and 22
problem-solving transfer idea units to the Taxonomy Table. Using SPSS1.1 to apply Cohen’s
Kappa, which is a standard measure for interrater reliability, we obtained a value of .59 for
interrater reliability on the transfer idea units and .74 for interrater reliability on the retention
idea units. The lower score of .59 is considered an acceptable value and indicates moderate
agreement on the retention idea units. The higher score of .74 indicate high interrater reliability
for the retention idea unit.
Assigning participant retention and transfer responses to idea units. The same process
that was used to determine reliability of assignment of idea units to the Taxonomy Table (figure
2) is being applied to assigning participant responses to the problem-solving strategies retention
and transfer questionnaire. This researcher and the research conducting related studies using the
computer game SafeCracker and identical problem-solving strategies retention and transfer
questionnaires are independently assigning participant responses to idea units. For the other
study (see Shen, in preparation), 466 statements are being coded. For this study, 1432 statements
are being coded. Following coding, Cohen’s Kappa will be run to determine interrater reliability.
Weighting participant responses. The cognitive process dimension contains six categories
(the columns in the table): Remember, Understand, Apply, Analyze, Evaluate, and Create. The
continuum underlying the cognitive process dimension is assumed to be cognitive complexity, as
Navigation Maps and Problem Solving: revised 5/18/05
98
the mention moves from left to right. (e.g., Understand is believed to be more cognitively
complex than Remember).
The knowledge dimension (the rows in the table) contains four categories: Factual
Knowledge, Conceptual Knowledge, Procedural Knowledge, and Metacognitive Knowledge.
These categories are assumed to lie along a continuum from concrete (Factual) to abstract
(Metacognitive). The Conceptual and Procedural categories overlap in terms of abstractness,
with some procedural knowledge being more concrete than the most abstract conceptual
knowledge (p. 5). Factual knowledge is knowledge of discrete, isolated content elements—bits
of information (p. 27) Conceptual knowledge is knowledge of “more complex, organized
knowledge forms” (p. 27).
In this study, participant responses to the problem-solving strategies retention and
transfer questionnaire are assigned to idea units which are in turn assigned to a cell in the
Taxonomy Table (Figure 2). Therefore, participant responses ultimately end up as a cell entry on
the Taxonomy Table. According to Anderson et al. (2001), the cognitive processes present in the
Taxonomy Table describe the range of cognitive activities in constructivist learning and that
constructivist learning is equivalent to meaningful learning. As discussed in chapter 2,
meaningful learning is defined as deep understanding of the material, which includes attending
to important aspects of the presented material, mentally organizing it into a coherent cognitive
structure, and integrating it with relevant existing knowledge (Mayer & Moreno, 2003).
Meaningful learning is reflected in the ability to apply what was taught to new situations;
problem solving transfer.
Anderson et al. (2001) further argue that, students often learn facts and ideas but not how
to apply, or transfer, them to novel situations. These students do not understand the information
Navigation Maps and Problem Solving: revised 5/18/05
99
at a deep level that would allow them to integrate or systematically organize their knowledge.
This type of shallow knowledge is referred to as inert knowledge and is of less educational value
than deeper levels of understanding. The cognitive process dimension of the Taxonomy Table
(figure 2) is organized from processes that could occur with shallow understanding (the left
column) to processes that could only occur with deep understanding (the right-most column). In
terms of the knowledge dimension of the Taxonomy Table (figure 2), the taxonomy was
designed to capture the growing level of complexity of knowledge while maintaining a
parsimony of knowledge categories. The four knowledge categories (Factual, Conceptual,
Procedural, and Metacognitive) represent increasing more complex knowledge.
Based on the acknowledgement that moving from left to right on the Taxonomy Table
and moving from top to bottom on the table represent increasingly more complex, deeper, and
more meaningful knowledge and processes, this researcher, along with the researcher from the
related SafeCracker study (see Shen, in preparation) have proposed a modification of the
procedure by Mayer and Moreno (see for example, Mayer, 2001; Mayer et al, 2003; Mayer &
Moreno, 1998; Mayer et al., 2003; Moreno & Mayer, 2000, 2004) and Chen (in preparation). We
propose that, instead of assigned a value of one to all participant responses, the value of the
response should be based on it’s location in the Taxonomy Table. Figure 3 shows the Taxonomy
Table with values added to the matrix. The top left cell, which represents application of the
shallowest knowledge, would receive a value of 1 and the bottom right cell, which represents
application the deepest knowledge, would receive a value of 9.
Figure 3: Taxonomy Table with knowledge values
The
Knowledge
The cognitive process dimension
1.
2.
3.
4.
5.
6.
Navigation Maps and Problem Solving: revised 5/18/05
Dimension
100
Remember
Understand
Apply
Analyze
Evaluate
Create
1
2
3
4
5
6
2
3
4
5
6
7
3
4
5
6
7
8
4
5
6
7
8
9
A.
Factual
Knowledge
B.
Conceptual
Knowledge
C.
Procedural
Knowledge
D.
MetaCognitive
Knowledge
The values in the Taxonomy Table (Figure 3) were established through discussions
between the two researchers. A number of questions needed to be addressed for determination of
values. One of the first questions was, “if we assign values of 1 to 6 to the top row, what should
the value be of the leftmost cell in the second row?” Ultimately, we decided that the first column
in the second row (representing Remembering Conceptual Knowledge) could be of equivalent
value to the second column of the first row (Understanding Factual Knowledge). This created a
pattern of the value of a cell being equivalent of the cell one above and one to the right.
Therefore, the first column, third row and the second column, second row and the third column
Navigation Maps and Problem Solving: revised 5/18/05
101
first row would have equivalent values. The researchers determined this value system fairly
represented the type of knowledge and the cognitive processes within the matrix.
The final determination for the value system was whether, by using the system just
described, the value of the bottom right cell could be 9 times more valuable than the value of the
top left cell. In our opinion, we believe that the ability to create (column) metacognitive
knowledge (row), i.e., the bottom right cell, could indeed be 9 times more valuable than the
ability to remember (column) facts (row), i.e., the top left cell.
Procedure for the Pilot Study
The pilot study began with introducing the participants to the objective of the
experiment, describing the experiment as an examination of methods that might help student
performance when using a video game for learning, but not discussing the issue of navigation
maps. Next participants and researcher signed a consent form and participants were assigned a
three-digit number that had been randomly generated prior to the study.
Demographic and self-regulation questionnaires.
Following the brief introduction, participants asked to fill out the demographic and selfregulation questionnaires (Appendices B and D, respectively). They were told they would have 8
minutes.
Introduction to the knowledge mapping software.
Following administration of the demographic and self-regulation questionnaires,
participants were asked to start the knowledge mapping software. Once started, knowledge
mapping was explained. Then the interface was explained by describing function of the Add
Concepts menu button and the three on-screen buttons. Participants were asked to click on a
Navigation Maps and Problem Solving: revised 5/18/05
102
concept to add it to the screen. Participants were then prompted to move the concept around.
Next participants were asked to add a few more concepts.
Participants were then asked to click the Link button and told that would switch them
to a mode that would allow links to be created between concepts. Participants were told to click
and drag from on concept to another. That caused a dialog box to open. Participants were
prompted to click on the dialog box’s pull-down menu, to see a list of available links.
Participants were then prompted to click on a link, which caused the dialog box to close and an
arrow to be drawn from one concept to the other.
Participants were asked to create several more links, after which they were told to click
the third mode button, the Erase button. Next, participants were asked to click on the words of a
link and saw the link disappear. Next they were prompted to click on a concept that had at least
one link connected to it and watched as both the concept and its links disappeared. Participants
were reminded that once they entered a mode (Move, Link, Erase), all they could do is that
mode. And they were told there was no undo button on the software. So if they accidentally
deleted a concept and all its links, they would need to recreate them. It was suggested they
change to link or move mode, as soon as they were done erasing items. Participants were asked if
they understood how to use the software. Upon receiving a positive response, they were shown
how to exit the software.
Introduction of the Game SafeCracker.
Participants were told they would next learn the game and were prompted to open
SafeCracker by clicking on an icon. Participants were then guided through entering the game,
finding the mansion (the main game play area of the game), entering the mansion, searching the
Navigation Maps and Problem Solving: revised 5/18/05
103
first room, and opening one safe. All participants received the same instruction, by use of a script
(Appendix G).
Introduction to the Navigation Map.
Those in the treatment group were next introduced to the navigation map and given
instruction in its use and strategies for its use for navigation. To ensure equivalent learning by all
those in the treatment group a script was utilized (Appendix H). The script included two types of
instruction. First was instruction on how to read the map. Second was instruction on how to
navigate using the map. Those in the control group were given simple guidelines for navigation
(Appendix I).
First Game.
After players were given instruction on navigating the environment, they were given their
first Task Completion Form (Appendix C), which listed the two of the three rooms involved in
the study and the safes they would need to open in those rooms. They were told to mark off safes
as they opened them. They were then prompted to open a game already in progress. Once the
game was open, they were then told the names of the three rooms involved in the first game and
told to take note that only two rooms and their safes were listed on the Task Completion Form.
Participants were told that they were currently in the third room (the Reception Room) and the
safes for that room had already been opened for them and the safes’ contents were in their
inventory.
Those in the treatment group were then handed their navigation map for the first game.
They were told that the shaded rooms for the first game were the same three rooms that were
shaded on the learning map. Both groups were reminded to not forget to look at objects in the
Navigation Maps and Problem Solving: revised 5/18/05
104
rooms, including the room they were currently in, the Reception Room. Finally, participants
were told they would have 15 minutes to find and open the safes and told to begin.
Creating the First Knowledge Map (intermediate test).
After fifteen minutes, participants were prompted to safe their game and exit
SafeCracker. They were then told to start the Knowledge Mapping software. After asking
whether they had any questions, participants were told they would have 7 minutes to create a
knowledge map and told to begin.
The First Problem-Solving Strategies Questionnaire (intermediate test).
Participants were handed the Problem-Solving Strategies questionnaire and told how to
fill it out. They were told the questionnaire was four pages long and contained two questions.
They were told each question involved two pages with the first question beginning on page 1 and
the second question beginning on page 2. They were further told to start on question 1 and to not
go to question 2 until told to do so. Last, they were told they would be given two minutes per
question and then told to begin.
Second Game.
Upon collecting the Problem-Solving Strategies questionnaires, participants were
prompted to restart SafeCracker and to open a different game that was already in progress. While
the program was opening up, participants were handed their second Task Completion Form
(Appendix C). They were told that one of the rooms was a room included on the first task; the
Technical Design Room. They were told that they would need to open the safes in that room
even if they had opened them in the first game. Those in the Treatment group were handed the
navigation map for the second game, which included the darkened rooms of that game.
Navigation Maps and Problem Solving: revised 5/18/05
105
Once the game was started and the appropriate game in progress was opened, participant
were told that the safes from the rooms in the first game that weren’t in the second game were
opened and their contents added to their inventory. Participants were reminded that they would
have 15 minutes for this game and were told to begin.
Second Knowledge Map and Second Problem-Solving Strategies Questionnaire.
After 15 minutes were up, participants were prompted to save their game and to exit
SafeCracker. They were next prompted to restart the Knowledge Mapping software and were
given 7 minutes to create their second knowledge map. Last, they were given their second
problem-solving strategies questionnaire, which was identical to the first problem-solving
strategies questionnaire and prompted to respond one question at a time, as with the prior
questionnaire. They were given a total of 4 minutes for the questionnaire; two minutes per
question.
Debriefing.
Upon completion of the second problem-solving strategies questionnaire, participants
were told the study was over. They were asked what they thought of the game and if it was
similar to games they’ve played or games they liked. If appropriate, they were asked what types
of games, and even specific games, they liked. They were also asked if they had any questions.
Finally, participants were told they could continue playing the game if they were interested.
Timing Chart for Pilot Study
Table 2 lists the activities encompassing the pilot study and the times allocated with each
activity, and ending with total time.
Table 2: Time Chart of the Pilot Study
ACTIVITY
TIME ALLOCATION
Navigation Maps and Problem Solving: revised 5/18/05
Introduction
106
3 minutes
And study related paperwork
Self-regulation and demographic questionnaires
8 minutes
Introduction to knowledge mapping software
10 minutes
Game introduction
15 minutes
Game playing (first 3 rooms)
15 minutes
And Task Completion Form
Knowledge map (intermediate)
7 minutes
Problem-solving strategy questions (intermediate)
4 minutes
Game playing (second 3 rooms)
15 minutes
And Task Completion Form
Knowledge map (post)
7 minutes
Problem-solving strategy questions (post)
4 minutes
Debriefing
3 minutes
TOTAL
91 minutes
Optional additional game time
Up to 30 minutes
Results of the Pilot Study
Overall, the instruments and procedures in the pilot study worked well. But there was
some need for modification and improvement of some of the instructions given to participants.
The first modification involved the amount of time allotted to participants for filling out the
demographic and self-regulation forms. Participants were told they had 8 minutes to fill out the
forms. While both participants completed both form well within the 8 minutes alotted, comments
from one of the participants indicated that unnecessary stress had been added by feeling time was
Navigation Maps and Problem Solving: revised 5/18/05
107
limited. It was decided that, for the main study, participants would not be told how much time
they had, but would be prompted to finish up soon, if time was running out. As expected, both
participants in the pilot study finished within the 8 minute time frame. In addition, it was
determined that the time allotted for filling out the demographic and self-regulation forms could
be reduced from 8 minutes to 7 minutes. This revision was made for the main study.
Adjustments to the knowledge mapping instruction. There were a number of small
problems discovered with the introduction of knowledge mapping. The first problem was the
introduction of extraneous cognitive load. Extraneous load refers to the cognitive load imposed
by unnecessary materials (Harp & Mayer, 1998; Mayer, Heiser, & Lonn, 2001; Moreno &
Mayer, 2000; Renkl & Atkinson, 2003; Schraw, 1998). In the pilot study, participants were
asked to open the software. Once opened, participants were told what knowledge mapping is.
This explanation took approximately one minute. Because the software was open, participants
were attending to the software while, at the same time, receiving information on knowledge
mapping. This imposed unnecessary, or extraneous, cognitive load. It was decided that, for the
pilot study, participants would be told about knowledge mapping and then prompted to start the
software.
Another important problem with the explanation of knowledge mapping was with the
examples given for types of links. Three types of knowledge map links were described: temporal
links, causal links, and simple relational links. In the pilot study, three random examples were
given. It was decided for the main study that all three examples should be within the same
domain. To support this, since all participants would be at, or had been at, the University of
Southern California (USC), the domain was for the knowledge mapping instruction would be
USC and all three link example would be related to USC. It was also determined that knowledge
Navigation Maps and Problem Solving: revised 5/18/05
108
mapping instruction could be reduced from the 10 minutes allotted for the pilot study to just 8
minutes for the main study.
Also a number of small changes were made to the knowledge mapping instructions. In
particular, a strategy component was added. In the main study, participants would be explicitly
told that EVERY concept was applicable to the game SafeCracker. The following instruction
was also added for the pilot study: “A recommended strategy is to begin your knowledge map by
opening the ‘add concept’ pull-down menu and clicking on every link. Then move the links
around so you can see all of them. Then switch to link mode and start making connections.”
Adjustments to the SafeCracker instructions. A number of small flaws were found with
the script for the SafeCracker instruction. While too numerous to mention all the problems, one
example is how subjects were introduced to panning their view within the game environment. In
the original script, as part of the panning instructions, participants were told to “click the mouse
in middle of the screen and don’t let go.” While this seems an obvious, explicit command,
participants varied in where on the screen they clicked, including toward the bottom or to the far
right side. A participant also asked, “Which middle? The middle of the monitor or the middle of
the main window on the interface.” For the main study, the script was changed to “click the
mouse one or two inches to the right of the phone’s hand piece and don’t let go.” This seemed to
alleviate the problem found in the pilot study.
As with the knowledge mapping instruction, it was determined that strategy instruction
needed to be added to the SafeCracker instruction. It was also noticed that both participants in
the pilot study forgot to search for clues. The following is an example of strategy instruction
added to the instructions for the main study: “Go ahead and click on it. Notice the diagrams.
These might be important for opening a safe. You might want to write them down later, when
Navigation Maps and Problem Solving: revised 5/18/05
109
you start playing the game.” The following search instructions, reminders, and strategies were
added to the end of the instructions for the treatment group in the main study.
Once you have a plan for how you will navigate to and from rooms, then you
would begin moving around, collecting items and attempting to open safes.
REMEMBER, IT IS VERY IMPORTANT THAT YOU LOOK AT ALL
THE ITEMS IN ROOMS, TO FIND CLUES THAT MIGHT HELP OPEN
SAFES. Not everything gets added to your inventory. You may need to write
things down.
The following search instructions, reminders, and strategies were added to the
instructions for the control group in the main study.
Once you have a plan for how you will navigate to and from rooms, then you
would begin moving around, collecting items and attempting to open safes.
REMEMBER, IT IS VERY IMPORTANT THAT YOU LOOK AT ALL
THE ITEMS IN ROOMS, TO FIND CLUES THAT MIGHT HELP OPEN
SAFES. Not everything gets added to your inventory. You may need to write
things down.
An important change was the addition of time added for navigation map
training for the treatment group. Originally, 15 minutes was allotted to SafeCracker
instruction. While this was enough time for the control group, the treatment group
needed more time. It was determined that 8 extra minutes were needed for map
instructions in the main study.
Adjustments to the problem-solving strategies questionnaire instructions. The next
change involved the problem-solving strategies questionnaire. In the pilot study, one participant
Navigation Maps and Problem Solving: revised 5/18/05
110
switched to the second question before the two minutes allotted for answering the first question
were up. It was determined that, for the main study, participants would be explicitly told “do not
go to the second question until told to do so. Continue to work on the first question for the full
two minutes. And once I tell you to go to the second question, do not return to the first question;
stay on the second question.”
Adjustment to the task completion form. One of the safes listed in the Task Completion
Form is the Strongbox in the Storeroom off of the Technical Design Room. This room and safe
appears on the task completion form for both tasks (Task 1 and Task 2). While this is an accurate
description of the safe and its location, the strongbox is inside a drawer in a file cabinet. This
confused participants in the pilot study. For the main study, the text on the Task Completion
Form was changed from “Strongbox (in storeroom)” to “Strongbox (file cabinet in storeroom).
Since this safe appeared on both task completion forms, the text was changed on both forms (see
Appendix C).
Running the pilot study resulted in confirming the validity of all instruments. All
instruments worked as expected, none were extraneous, and no addition instruments were
needed. However, modifications were made to the Task Completion form (Appendix C) and the
SafeCracker instructions (Appendix G, H, and I). Changes were also made to the study timeline,
because it was discovered that some processes could happen more quickly, while one instruction
(map instruction) needed additional time. The pilot study timeline encompassed 91 minutes. The
main study timeline would encompass 96 minutes. It should be noted that all instruments listed
in the appendices reflect the changes made to the instruments after the pilot study. For final
versions of modified forms, see Appendix C, G, H, and I.
Procedure for the Main Study
Navigation Maps and Problem Solving: revised 5/18/05
111
The main study began with introducing the participants to the objective of the
experiment, describing the experiment as an examination of methods that might help student
performance when using a video game for learning, but not discussing the issue of navigation
maps. Next participants and researcher signed a consent form and participants were assigned a
three-digit number that had been randomly generated prior to the study.
Demographic and self-regulation questionnaires.
Following the brief introduction, participants asked to fill out the demographic and selfregulation questionnaires (Appendices B and D, respectively). They were given 7 minutes to fill
out the forms, but were not told there was a time limit. If the 7 minute time limit was imminent,
and it appeared a participant might not finish in time, that participant was told he or she only had
a minute or two left. This only happened once during the study (one participant). Most
participants finished filling out the two questionnaires within 5 minutes. If a participant asked
what a game term meant, he or she was prompted to enter a zero for their Likert response.
Introduction to the knowledge mapping software.
Following administration of the demographic and self-regulation questionnaires,
participants were asked to start the knowledge mapping software. Before starting the software,
knowledge mapping was explained, by using the University of Southern California as a domain.
After numerous concepts were listed, such as school, university, classroom, book, teacher,
student, sorority, fraternity, study, and party, three link examples were given. For an example of
a temporal link, the phrase “Study before tests” was given, where before is the temporal link. For
an example of a causal link, the phrase “Studying improves grades” was given, where improves
is the causal link. For a simple relational link, the phrase “Classrooms contain book” was given,
where contains is the relational link. So that participants understood the reason for creating a
Navigation Maps and Problem Solving: revised 5/18/05
112
knowledge map, they were that research has provided strong evidence that a person’s ability to
create a knowledge map is directly related to that persons understanding of a subject matter; that
the more accurate and the more complete the knowledge map, the better a person understands a
domain.
Participants were then prompted to start the knowledge mapping software. Once
started, the interface was explained, by describing function of the Add Concepts menu button
and the three on-screen buttons. Participants were asked to click on a concept to add it to the
screen. Participants were then asked to add three more concepts “for a total of four concepts.”
Participants were next prompted to open the Add Concept pull-down menu and to take note that
the four concepts they selected were grayed out in the menu. It was explained that a concept
could only be added once, since it could have as many links going to it or coming from it as
desired. Participants were also told that every concept in the pull-down menu applied to the game
SafeCracker, therefore every concept could be used in a knowledge map.
Next, participants were then prompted to move the four concepts around to form a
large box. Once completed, the three Mode buttons near the bottom of the screen were explained,
as well as the display box to the right of the button. It was explained that the reason they (the
participant) were able to move the concepts around was because they were in Move mode, as
indicated by the word Move in the display box. The participants were told that once they clicked
on a Mode button, they would remain in that mode until they clicked another mode button. They
were then asked to click the Link mode button and were pointed to the display box to see that it
now showed the word Link.
Participants were told to click in the middle of one concept and drag to the middle of
another concept before letting go of the mouse, in order to generate a link. Once they did so, and
Navigation Maps and Problem Solving: revised 5/18/05
113
the link dialog box opened, participants were told to click on the dialog box’s pull-down menu
and look at the choice given for links, such as contains, leads to, or requires. They were asked to
pick a link and not to worry about the appropriateness of the link. They were told that, for now,
the researcher was only concerned that they learn to use the software and didn’t care whether
they created an accurate knowledge map.
After participants successfully created the first link, they were prompts to add at least 5
more links. They were told that, for the purposes of instruction, one concept needed to have only
one link, either going to it or coming from it, and that the rest of the concepts could have as
many links attached to them as desired. Once enough links were added by all participants, they
(the participants) were asked to click on the Erase mode button, to switch to Erase mode. They
were prompted to look at the display window to see that it now displayed the word Erase.
Participants were told not to click on anything until explicitly told to do so. They were
then told that the way to erase a link was to click on the words attached to the link (such as
causes or leads to). Participants were then prompted to click on the word of the link that was
connected to the concept that only had one link. After that, participants were told that the way to
erase a concept was to click directly on the concept. They were then prompted to click on the
concept that no longer had a link. Then, individually, the researcher told each participant a
specific concept to click on. The concept selected was whichever link had a large number of
links connected to it. Upon clicking the concept, the link and all its links were erased. Some
participants were so surprised that they made an audible sound of shock. Participants were told
that the software had no undo button, and that if they accidentally erased a concept and all its
links, they would need to recreate them. They were told that the moment they were done erasing,
Navigation Maps and Problem Solving: revised 5/18/05
114
they should immediately switch to either Move or Link mode, to avoid erasing accidentally.
They were then prompted to switch to Move mode.
Because participants had begun with 4 concepts and had erased two of them, each now
only had two concepts on the screen. In almost all cases, there was also a link between those two
remaining concepts. If there wasn’t, participants were prompted to add a link, but switching to
Link mode, and then prompted to switch back to Move mode. Then participants were asked to
click and drag one of the concepts and were shown that, as a concept was moved, its links moved
with it. They were reminded that all the concepts in the software applied to SafeCracker and that
a recommended strategy for creating a knowledge map was to begin by adding all the concepts to
the screen and then move the concepts around so they could begin creating links. They were told
that, as the screen got crowded, they could move concepts around and their links would go with
them. Participants were then asked if they understood how to use the software. Upon receiving a
positive response, they were shown how to exit the software and how to safe their file when
asked to do so later.
Introduction of the Game SafeCracker.
A script was used, to ensure equivalent learning by all participants (Appendix G). The
procedures for teaching SafeCracker were the same as those used in the pilot study, with the
addition of reminders to all participants to check for clues as they played the game and that not
all clues were added to the inventory, so they might need to write some things down on the
scratch paper they were provided. They were also told they could get as many pieces of scratch
paper as desired.
Introduction to the Navigation Map.
Navigation Maps and Problem Solving: revised 5/18/05
115
From the pilot study, an additional 9 minutes had been added to the treatment group for
introducing those in the treatment group to map reading, navigation, as well as instructions in
map use and strategies. To ensure equivalent learning by all those in the treatment group a script
was utilized (Appendix H). The script included three types of instruction. First was instruction on
how to read the map. Second was instruction on how to navigate using the map. Third were
strategy suggestions. Those in the control group were given simple guidelines for navigation and
strategies for navigation (Appendix I).
First Game.
After players were given instruction on navigating the environment, they were given their
first Task Completion Form (Appendix C), which listed the two of the three rooms involved in
the study and the safes they would need to open in those rooms. They were told to mark off safes
as they opened them. They were then prompted to open a game already in progress. Once the
game was open, they were then told the names of the three rooms (Reception room, Small
Showroom, and Technical Design room) involved in the first game and told to take note that
only two rooms (Small Showroom and Technical Design room) and their safes were listed on the
Task Completion Form. Participants were told that they were currently in the third room
(Reception Room) and the safes for that room had already been opened for them and the safes’
contents were in their inventory.
Those in the treatment group were then handed their navigation map for the first game.
They were told that the shaded rooms for the first game were the same three rooms that were
shaded on the learning map. Both groups were reminded to not forget to look at objects in the
rooms, including the room they were currently in, the Reception Room. Finally, participants
were told they would have 15 minutes to find and open the safes and told to begin.
Navigation Maps and Problem Solving: revised 5/18/05
116
Creating the First Knowledge Map (intermediate test).
After fifteen minutes, participants were prompted to safe their game using their number
hyphen one. For example, if the participant’s number was 803, the filename would be 803-1.
Participants were then prompted to exit SafeCracker. They were then told to start the Knowledge
Mapping software. After asking whether they had any questions, participants were told they
would have 7 minutes to create a knowledge map and told to begin. After 7 minutes, participants
were prompted to safe their file as their number hyphen one (e.g., 803-1) and to exit the
software.
The First Problem-Solving Strategies Questionnaire (intermediate test).
As with the pilot study, participants were given the first Problem-Solving Strategies
questionnaire for the first game and told how to fill it out. They were told to stay on the first
question until told to go to the second question and that once they were on the second question
they were to remain there and not go back to the first questions. There were given four minutes
for the questionnaire, 2 minutes per question.
Second Game.
Procedures for the second game were the same as for the second game of the pilot study.
Participants started the game and were told to open a different game already in progress. They
were also handed the Task Completion form for the second task (Appendix C). This form had
been modified from the pilot study. The wording for the Strongbox safe in the Technical Design
room was changed from “Strongbox (in storeroom)” to “Strongbox (file cabinet in storeroom)”
Those in the Treatment group were handed the navigation map for the second game, which
included the darkened rooms of the game. Participants were reminded that they would have 15
minutes for this game and were told to begin.
Navigation Maps and Problem Solving: revised 5/18/05
117
Second Knowledge Map and Second Problem-Solving Strategies Questionnaire.
After 15 minutes were up, participants were prompted to save their game with their
number hyphen two. For example, if a participant’s number was 803, the file name would be
803-2. Participants were then asked to exit SafeCracker. They were next prompted to restart the
Knowledge Mapping software and were given 7 minutes to create their second knowledge map.
After 7 minutes, participants were prompted to safe their file as their number hyphen one (e.g.,
803-1) and to exit the software. Last, participants were given their second problem-solving
strategies questionnaire, which was identical to the first problem-solving strategies questionnaire
and prompted to respond one question at a time, as with the prior questionnaire. They were given
a total of 4 minutes for the questionnaire; two minutes per question.
Debriefing.
Upon completion of the second problem-solving strategies questionnaire, participants
were told the study was over. They were asked what they thought of the game and if it was
similar to games they’ve played or games they liked. If appropriate, they were asked what types
of games, and even specific games, they liked. They were also asked if they had any questions.
Finally, participants were told they could continue playing the game if they were interested.
Timing Chart for Main Study
Table 3 lists the activities encompassing the main study and the times allocated with each
activity, and ending with total time.
Table 3: Time Chart of the Main Study
ACTIVITY
TIME ALLOCATION
Introduction
3 minutes
And study related paperwork
Navigation Maps and Problem Solving: revised 5/18/05
Self-regulation and demographic questionnaires
7 minutes
Introduction to knowledge mapping software
8 minutes
Game introduction
15 minutes
Introduction of map reading to treatment group
8 minutes
Game playing (first 3 rooms)
15 minutes
And Task Completion Form
Knowledge map (intermediate)
7 minutes
Problem-solving strategy questions (intermediate)
4 minutes
Game playing (second 3 rooms)
15 minutes
And Task Completion Form
Knowledge map (post)
7 minutes
Problem-solving strategy questions (post)
4 minutes
Debriefing
3 minutes
TOTAL
96 minutes
Optional additional game time
Up to 30 minutes
118
Navigation Maps and Problem Solving: revised 5/18/05
119
CHAPTER IV: ANALYSIS AND RESULTS
Descriptive statistics (.e.g., means and standard deviations) will be used throughout the
study to describe all measures. The relationship of self-regulation and of presence or absence of a
navigation map to task difficulty, as measured by task completion, will be analyzed using
correlation analyses.
Hypothesis 1: Navigation maps will produce a significant increase in content
understanding compared to the control group.
A t-test will be used to compare the effect of a navigation map on content understanding
as compared to no map.
Hypothesis 2: Navigation maps will produce a significant increase in problem solving
strategy retention compared to the control group.
A t-test will be used to compare the effect of a navigation map on problem strategy
retention as compared to no map.
Hypothesis 3: Navigation maps will produce a significant increase in problem solving
strategy transfer compared to the control group.
A t-test will be used to compare the effect of a navigation map on problem strategy
transfer as compared to no map.
Hypothesis 4: There will be no significant difference in self-regulation between the
navigation map group and the control group. However, it is expected that higher levels of selfregulation will be associated with better performance.
Pearson’s correlation will be used to assess the effect of the four self-regulation variables
(planning, self-checking/monitoring, self-efficacy, and effort) on the navigation map group as
compared to the control group.
Navigation Maps and Problem Solving: revised 5/18/05
120
Hypothesis 5: Navigation maps will produce a significantly greater amount of optional
continued game play compared to the control group.
A t-test will be used to compare the effect of a navigation map on continued motivation
as compared to no map.
Navigation Maps and Problem Solving: revised 5/18/05
121
REFERENCES
Adams, P. C. (1998, March/April). Teaching and learning with SimCity 2000 [Electronic
Version]. Journal of Geography, 97(2), 47-55.
Alessi, S. M. (2000). Simulation design for training and assessment. In H. F. O’Neil, Jr. & D. H.
Andrews (Eds.), Aircrew training and assessment (pp. 197-222). Mahwah, NJ: Lawrence
Erlbaum Associates.
Alexander, P. A. (1992). Domain knowledge: Evolving themes and emerging concerns.
Educational Psychologist, 27(1), 33-51.
Allen, R. B. (1997). Mental models and user models. In M. Helander, T. K. Landauer & P.
Prabhu (eds.), Handbook of Human Computer Interaction: Second, Completely Revised
Edition (pp. 49-63). Amsterdam: Elsevier
Anderson, R. C., & Pickett, J. W. (1978). Recall of previously recallable information following a
shift in perspective. Journal of Verbal Learning and Verbal Behavior, 17, 1-12.
Asakawa, T., Gilbert, N. (2003). Synthesizing experiences: Lessons to be learned from Internetmediated simulation games. Simulation & Gaming, 34(1), 10-22.
Atkinson, R. K., Derry, S. J., Renkl, A., & Wortham, D. (2000). Learning from examples:
Instructional principles from the worked examples research. Review of Educational
Research, 70(2), 181-214.
Atkinson, R. K., Renkl, A., Merrill, M. M. (2003). Transitioning from studying examples to
solving problems: Effects of self-explanation prompts and fading worked-out steps.
Journal of Educational Psychology, 95(4), 774-783.
Ausubel, D. P. (1963). The psychology of meaningful verbal learning. New York: Grune and
Stratton.
Navigation Maps and Problem Solving: revised 5/18/05
122
Baddeley, A. D. (1986). Working memory. Oxford, England: Oxford University Press.
Baddeley, A. D., & Logie, R. H. (1999). Working memory: The multiple-component model. In
A. Miyake & P. Shah (Eds). Models of working memory: Mechanisms of active
maintenance and executive control (pp. 28-61). Cambridge, England: Cambridge
University Press.
Baker, D., Prince, C., Shrestha, L., Oser, R., & Salas, E. (1993). Aviation computer games for
crew resource management training. The International Journal of Aviation Psychology,
3(2), 143-156.
Baker, E. L., & Mayer, R. E. (1999). Computer-based assessment of problem solving. Computers
in Human Behavior, 15, 269-282.
Baker, E. L. & O’Neil, H. F., Jr. (2002). Measuring problem solving in computer environments:
current and future states. Computers in Human Behavior, 18, 609-622.
Banbury, S. P., Macken, W. J., Tremblay, S., & Jones, D. M. (2001, Spring). Auditory
distraction and short-term memory: Phenomena and practical implications. Human
Factors, 43(1), 12-29.
Barab, S. A., Bowdish, B. E., & Lawless, K. A. (1997). Hypermedia navigation: Profiles of
hypermedia users. Educational Technology Research & Development, 45(3), 23-41.
Barab, S. A., Young, M. F., & Wang, J. (1999). The effects of navigational and generative
activities in hypertext learning on problem solving and comprehension. International
Journal of Instructional Media, 26(3), 283-309.
Baylor, A. L. (2001). Perceived disorientation and incidental learning in a web-based
environment: Internal and external factors. Journal of Educational Multimedia and
Hypermedia, 10(3), 227-251.
Navigation Maps and Problem Solving: revised 5/18/05
123
Benbasat, I., & Todd, P. (1993). An experimental investigation of interface design alternatives:
Icon vs. text and direct manipulation vs. menus. International Journal of Man-Machine
Studies, 38, 369-402.
Berson, M. J. (1996, Summer). Effectiveness of computer technology in the social studies: A
review of the literature. Journal of Research on Computing in Education, 28(4), 486499.
Berylne, D. E. (1960). Conflict, arousal, and curiosity. New York: McGraw-Hill.
Betz, J. A. (1995/1996). Computer games: Increase learning in an interactive multidisciplinary
environment. Journal of Educational Technology Systems, 24(2), 195-205.
Bong, M. (2001). Between- and within-domain relationships of academic motivation among
middle and high school students: Self-efficacy, task-value, and achievement goals.
Journal of Educational Psychology, 93(1), 23-34.
Brougere, G. (1999, June). Some elements relating to children’s play and adult
simulation/gaming. Simulation & Gaming, 30(2), 134-146.
Brown, D. W., & Schneider, S. D. (1992), Young learners’ reactions to problem solving
contrasted by distinctly divergent computer interfaces. Journal of Computing in
Childhood Education, 3(3/4), 335-347.
Brozik, D., & Zapalska, A. (2002, June). The PORTFOLIO GAME: Decision making in a
dynamic environment. Simulation & Gaming, 33(2), 242-255.
Brunken, R., Plass, J. L., & Leutner, D. (2003). Direct measurement of cognitive load in
multimedia learning. Educational Psychologist 38(1), 53-61.
Brunning, R. H., Schraw, G. J., & Ronning, R R. (1999). Cognitive psychology and instruction
(3rd ed.). Upper Saddle River, NJ: Merrill.
Navigation Maps and Problem Solving: revised 5/18/05
124
Carroll, W. M. (1994). Using worked examples as an instructional support in the algebra
classroom. Journal of Educational Psychology, 86(3), 360-367.
Chalmers, P. A. (2003). The role of cognitive theory in human-computer interface. Computers in
Human Behavior, 19, 593-607.
Chen, H. H. (in preparation) A formative evaluation of the training effectiveness of a computer
game. Dissertation Proposal. University of Southern California.
Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations:
How students study and use examples in learning to solve problems. Cognitive
Science, 13, 145-182.
Chou, C., & Lin, H. (1998). The effect of navigation map types and cognitive styles on learners’
performance in a computer-networked hypertext learning system [Electronic Version].
Journal of Educational Multimedia and Hypermedia, 7(2/3), 151-176.
Chou, C., Lin, H, & Sun, C.-t. (2000). Navigation maps in hierarchical-structured hypertext
courseware [Electronic Version]. International Journal of Instructional Media, 27(2),
165-182.
Chuang, S.-h. (2003). The role of search strategies and feedback on a computer-based problem
solving task. Unpublished doctoral dissertation. University of Southern California.
Clark, R. E. (1999). The CANE model of motivation to learn and to work: A two-stage process
of goal commitment and effort [Electronic Version]. In J. Lowyck (Ed.), Trends in
Corporate Training. Leuven, Belgium: University of Leuven Press.
Clark, R. E. (2003a, February). Strategies based on effective feedback during learning. In H. F.
O’Neil (Ed.), What Works in Distance Education. Report to the Office of Naval Research
Navigation Maps and Problem Solving: revised 5/18/05
125
by the National Center for Research on Evaluation, Standards, and Student Testing (pp.
18-19).
Clark, R. E. (2003b, February). Strategies based on increasing student motivation: Encouraging
active engagement and persistence. In H. F. O’Neil (Ed.), What Works in Distance
Education. Report to the Office of Naval Research by the National Center for Research
on Evaluation, Standards, and Student Testing (pp. 20-21).
Clark, R. E. (2003b, February). Strategies based on providing learner control of instructional
navigation. In H. F. O’Neil (Ed.), What Works in Distance Education. Report to the
Office of Naval Research by the National Center for Research on Evaluation, Standards,
and Student Testing (pp. 14-15).
Clark, R. E. (2003d, March). Fostering the work motivation of teams and individuals.
Performance Improvement, 42(3), 21-29.
Clark, R. E. (Ed.).(2001). Learning from Media: Arguments, analysis, and evidence. Greenwich,
CT: Information Age Publishing.
Cobb, T. (1997). Cognitive efficiency: Toward a revised theory of media. Educational
Technology Research and Development, 45(4), 21-35.
Coffin, R. J., & MacIntyre, P. D. (1999). Motivational influences on computer-related affective
states. Computers in Human Behavior, 15, 549-569.
Corno, L., & Mandinach, E. B. (1983). The role of cognitive engagement in classroom learning
and motivation. Educational Psychologist, 18(2), 88-108.
Crookall, D., & Aria, K. (Eds.). (1995). Simulation and gaming across disciplines and cultures:
ISAGA at a watershed. Thousand Oaks, CA: Sage.
Navigation Maps and Problem Solving: revised 5/18/05
126
Crookall, D., Oxford, R. L., & Saunders, D. (1987). Towards a reconceptualization of
simulation. From representation to reality. Simulation/Games for Learning, 17, 147171.
Cross, T. L. (1993, Fall). AgVenture: A farming strategy computer game. Journal of Natural
Resources and Life Sciences Education, 22, 103-107.
Csikszentmihalyi, M. (1975). Beyond boredom and anxiety. San Francisco: Jossey Bass.
Csikszentmihalyi, M. (1990). Flow: The psychology of optimal performance. New York:
Cambridge University Press.
Cutmore, T. R. H., Hine, T. J., Maberly, K. J., Langford, N. M., & Hawgood, G. (2000).
Cognitive and gender factors influencing navigation in a virtual environment.
International Journal of Human-Computer Studies, 53, 223-249.
Daniels, H. L., & Moore, D. M. (2000). Interaction of cognitive style and learner control in a
hypermedia environment. International Journal of Instructional Media, 27(4), 369-383.
Davis, S., & Wiedenbeck, S. (2001). The mediating effects of intrinsic motivation, ease of use
and usefulness perceptions on performance in first-time and subsequent computer users.
Interacting with Computers, 13, 549-580.
Day, E. A., Arthur, W., Jr., & Gettman, D. (2001). Knowledge structures and the acquisition of a
complex skill. Journal of Applied Psychology, 86(5), 1022-1033.
de Jong, T., de Hoog, R., & de Vries, F. (1993). Coping with complex environments: The effects
of providing overviews and a transparent interface on learning with a computer
simulation. International Journal of Man-Machine Studies, 39, 621-639.
de Jong, T., & van Joolingen, W. R. (1998). Scientific discovery learning with computer
simulations of conceptual domains. Review of Educational Research, 68, 179-202.
Navigation Maps and Problem Solving: revised 5/18/05
127
deCharms, R. (1986). Personal Causation. New York: Academic Press.
Deci, E. L. (1975). Intrinsic Motivation. New York: Plenum Press.
Dekkers, J., & Donati, S. (1981). The interpretation of research studies on the use of simulation
as an instructional strategy. Journal of Educational Research, 74(6), 64-79.
Dempsey, J. V., Haynes, L. L., Lucassen, B. A., & Casey, M. S. (2002). Forty simple computer
games and what they could mean to educators. Simulation & Gaming, 43(2), 157-168.
Dias, P., Gomes, M. J., & Correia, A. P. (1999). Disorientation in hypermedia environments:
Mechanisms to support navigation. Journal of Educational Computing Research, 20(2),
93-117.
Dillon, A., & Gabbard, R. (1998, Fall). Hypermedia as an educational technology: A review of
the quantitative research literature on learner comprehension, control, and style. Review
of Educational Research, 63(3), 322-349.
Donchin, E. (1989). The learning strategies project. Acta Psychologica, 71, 1-15.
Druckman, D. (1995). The educational effectiveness of interactive games. In D. Crookall & K.
Aria (Eds.), Simulation and gaming across disciplines and cultures: ISAGA at a
watershed (pp. 178-187). Thousand Oaks, CA: Sage
Eberts, R. E., & Bittianda, K. P. (1993). Preferred mental models for direct-manipulation and
command-based interfaces. International Journal of Man-Machine Studies, 38, 769-785.
Eccles, J. S., & Wigfeld, A. (2002). Motivational beliefs, values, and goals. Annual Review of
Psychology, 53, 109-132.
Frohlich, D. M. (1997). Direct manipulation and other lessons. In M. Helander, T. K. Landauer
& P. Prabhu (eds.), Handbook of Human Computer Interaction: Second, Completely
Revised Edition (pp. 463-488). Amsterdam: Elsevier
Navigation Maps and Problem Solving: revised 5/18/05
128
Galimberti, C., Ignazi, S., Vercesi, P., & Riva, G. (2001). Communication and cooperation in
networked environment: An experimental analysis. CyberPsychology & Behavior,
4(1), 131-146.
Garris, R., Ahlers, R., & Driskell, J. E. (2002). Games, motivation, and learning: A research and
practice model. Simulation & Gaming, 33(4), 441-467.
Gevins, A., Smith, M. E., Leong, H., McEvoy, L., Whitfield, S., Du, R., & Rush, G. (1998).
Monitoring working memory load during computer-based tasks with EEG pattern
recognition methods. Human Factors, 40(1), 79-91.
Gredler, M.E. (1996). Educational games and simulations: a technology in search of a research
paradigm. In D. H. Jonassen (Ed.). Handbook of Research for Educational
Communications and Technology. (pp 521-540). New York: Simon & Schuster
Macmillan.
Green, C. S., & Bavelier, D. (2003, May 29). Action video game modifies visual selective
attention. Nature, 423, 534-537.
Greenfield, P. M., deWinstanley, P., Kilpatrick, H., & Kaye, D. (1996). Action video games and
informal education: Effects on strategies for dividing visual attention. In P. M. Greenfield
& R. R. Cocking (Eds.), Interacting with Video (pp. 187-205). Norwood, NJ: Ablex
Publishing Corporation.
Hannifin, R. D., & Sullivan, H. J. (1996). Preferences and learner control over amount of
instruction. Journal of Educational Psychology, 88, 162-173.
Harp, S. F., & Mayer, R. E. (1998). How seductive details do their damage: A theory of
cognitive interest in science learning. Journal of Educational Psychology, 90(3), 414434.
Navigation Maps and Problem Solving: revised 5/18/05
129
Harter, S. (1978). Effectance motivation reconsidered: Toward a developmental model. Human
Development, 1, 34-64.
Henderson, L., Klemes, J., & Eshet, Y. (2000). Just playing a game? Educational simulation
software and cognitive outcomes. Journal of Educational Computing Research, 22(1),
105-129.
Herl, H. E., Baker, E. L., & Niemi, D. (1996). Construct validation of an approach to modeling
cognitive structure of U.S. history knowledge. Journal of Educational Psychology, 89(4),
206-218.
Herl, H. E., O’Neil, H. F., Jr., Chung, G., & Schacter, J. (1999) Reliability and validity of a
computer-based knowledge mapping system to measure content understanding. Computer
in Human Behavior, 15, 315-333.
Hong, E., & O’Neil, H. F. Jr. (2001). Construct validation of a trait self-regulation model.
International Journal of Psychology, 36(3), 186-194.
Howland, J., Laffey, J., & Espinosa, L. M. (1997). A computing experience to motivate children
to complex performances [Electronic Version]. Journal of Computing in Childhood
Education, 8(4), 291-311.
Hubbard, P. (1991, June). Evaluating computer games for language learning. Simulation &
Gaming, 22(2), 220-223.
Jones, M. G., Farquhar, J. D., & Surry, D. W. (1995, July/August). Using metacognitive theories
to design user interfaces for computer-based learning. Educational Technology, 35(4),
12-22.
Navigation Maps and Problem Solving: revised 5/18/05
130
Kaber, D. B., Riley, J. M., & Tan, K.-W. (2002). Improved usability of aviation automation
through direct manipulation and graphical user interface design. The International
Journal of Aviation Psychology, 12(2), 153-178.
Kagan, J. (1972). Motives and development. Journal of Personality and Social Psychology, 22,
51-66.
Kalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2003). The expertise reversal effect.
Educational Psychologist, 38(1), 23-31.
Kalyuga, S., Chandler, P., & Sweller, J. (1998). Levels of expertise and instructional design.
Human Factors, 40(1), 1-17.
Kee, D. W., & Davies, L. (1988). Mental effort and elaboration: A developmental analysis.
Contemporary Educational Psychology, 13, 221-228.
Khoo, G.-s., & Koh, t.-s. (1998). Using visualization and simulation tools in tertiary science
education [Electronic Version]. The Journal of Computers in Mathematics and Science
Teaching, 17(1), 5-20.
Lee, J. (1999). Effectiveness of computer-based instructional simulation: A meta analysis.
International Journal of Instructional Media, 26(1), 71-85.
Leemkuil, H., de Jong, T., de Hoog, R., & Christoph, N. (2003). KM Quest: A collaborative
Internet-based simulation game. Simulation & Gaming, 34(1), 89-111.
Locke, E. A., & Latham, G. P. (1990). A theory of goal setting and task performance.
Englewood Cliffs, NJ: Prentice Hall.
Locke, E. A., Latham, G. P. (2002, Summer). Building a practically useful theory of goal setting
and task motivation: A 35-year odyssey. American Psychologist, 57(9), 705-717.
Malone, T. W. (1981). What makes computer games fun? Byte, 6(12), 258-277.
Navigation Maps and Problem Solving: revised 5/18/05
131
Malone, T. W., & Lepper, M. R. (1987). Making leraning fun: A taxonomy of intrinsic
motivation for learning. In R. E. Snow & M. J. Farr (Eds.). Aptitute, learning, and
instruction: Vol. 3. Conative and affective process analyses (pp. 223-253). Hillsdale,
NJ: Lawrence Erlbaum.
Malouf, D. (1987-1988). The effect of instructional computer games on continuing student
motivation. The Journal of Special Education, 21(4), 27-38.
Mayer, R. E. (1981). A psychology of how novices learn computer programming. Computing
Surveys, 13, 121-141.
Mayer, R. E. (1998). Cognitive, metacognitive, and motivational aspects of problem solving.
Instructional Science, 26, 49-63.
Mayer, R. E., & Chandler, P. (2001). When learning is just a click away: Does simple user
interaction foster deeper understanding of multimedia messages? Journal of Educational
Psychology, 93(2), 390-397.
Mayer, R. E., Heiser, J., & Lonn, S. (2001). Cognitive constraints on multimedia learning: When
presenting more material results in less understanding. Journal of Educational
Psychology, 93(1), 187-198.
Mayer, R. E., Mautone, P., & Prothero, W. (2002). Pictorial aids for learning by doing in a
multimedia geology simulation game. Journal of Educational Psychology, 94(1), 171185.
Mayer, R. E., & Moreno, R. (1998). A split-attention effect in multimedia learning: Evidence of
dual processing systems in working memory. Journal of Educational Psychology, 90(2),
312-320.
Navigation Maps and Problem Solving: revised 5/18/05
132
Mayer, R. E., & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia learning.
Educational Psychologist, 38(1), 43-52.
Mayer, R. E., Moreno, R., Boire, M., & Vagge, S. (1999). Maximizing constructivist learning
from multimedia communications by minimizing cognitive load. Journal of Educational
Psychology, 91(4), 638-643.
Mayer, R. E., & Wittrock, M. C. (1996). Problem-solving transfer. In D. C. Berliner & R. C.
Calfee (Eds.), Handbook of educational psychology (pp. 47-62). New York: Simon &
Schuster Macmillan.
McGrenere, J. (1996). Design: Educational electronic multi-player games—A literature review
(Technical Report No. 96-12, the University of British Columbia). Retrieved from
http://taz.cs.ubc.ca/egems/papers/desmugs.pdf
Moreno, R., & Mayer, R. E. (1999). Cognitive principles of multimedia learning: The role of
modality and contiguity. Journal of Educational Psychology, 91(2), 358-368.
Moreno, R., & Mayer, R. E. (2000a). A coherence effect in multimedia learning: The case of
minimizing irrelevant sounds in the design of multimedia instructional messages. Journal
of Educational Psychology, 92(1), 117-125.
Moreno, R., & Mayer, R. E. (2000b). Engaging students in active learning: The case for
personalized multimedia messages. Journal of Educational Psychology, 92(4), 724-733.
Moreno, R., & Mayer, R. E. (2002). Learning science in virtual reality multimedia environments:
Role of methods and media. Journal of Educational Psychology, 94(3), 598-610.
Mousavi, S. Y., Low, R., & Sweller, J. (1995). Reducing cognitive load by mixing auditory and
visual presentation modes. Journal of Educational Psychology, 87(2), 319-334.
Navigation Maps and Problem Solving: revised 5/18/05
133
Mwangi, W., & Sweller, J. (1998). Learning to solve compare word problems: The effect of
example format and generating self-explanations. Cognition and Instruction, 16(2), 173199.
Niemiec, R. P., Sikorski, C., & Walberg, H. J. (1996). Learner-control effects: A review of
reviews and a meta-analysis. Journal of Educational Computing Research, 15(2), 157174.
Noyes, J. M., & Garland, K. J. (2003). Solving the Tower of Hanoi: Does mode of presentation
matter? Computers in Human Behavior, 19, 579-592.
O’Neil, H. F., Jr. (1999). Perspectives on computer-based performance assessment of problem
solving: Editor’s introduction. Computers in Human Behavior, 15, 255-268.
O'Neil, H. F., Jr. (2002). Perspective on computer-based assessment of problem solving [Special
Issue]. Computers in Human Behavior, 18(6), 605-607.
O’Neil, H. F., Jr., & Abedi, J. (1996). Reliability and validity of a state metacognitive inventory:
Potential for alternative assessment. Journal of Educational Research, 89, 234-245.
O’Neil, H. F., Jr., Baker, E. L., Fisher, J. Y.-C. (2002, August 31). A formative evaluation of ICT
games. Manuscript: University of California, Los Angeles, National Center for
Research on Evaluation, Standards, and Student Testing (CRESST)
O’Neil, H. F., Jr., & Herl, H. E. (1998). Reliability and validity of a trait measure of selfregulation. Manuscript: University of California, Los Angeles, National Center for
Research on Evaluation, Standards, and Student Testing (CRESST).
Paas, F., Renkl, A., & Sweller, J. (2003). Cognitive load theory and instructional design: Recent
developments. Educational Psychologist, 38(1), 1-4.
Navigation Maps and Problem Solving: revised 5/18/05
134
Paas, F., Tuovinen, J. E., Tabbers, H., & Van Gerven, P. W. M. (2003). Cognitive load
measurement as a means to advance cognitive load theory. Educational Psychologist,
38(1), 63-71.
Park, O.-C., & Gittelman, S. S. (1995). Dynamic characteristics of mental models and dynamic
visual displays. Instructional Science, 23, 303-320.
Perkins, D. N., & Salomon, G. (1989). Are cognitive skills context bound? Educational
Researcher, 18, 16-25.
Pintrich, P. R., & DeGroot, E. V. (1990). Motivational and self-regulated learning components
of classroom academic performance. Journal of Educational Psychology, 82, 33-40.
Plotnick, E. (1999). Concept mapping: A graphical system for understanding the relationship
between concepts. Educational Media & Technology Yearbook, 24, 81-84.
Porter, D. B., Bird, M. E., & Wunder, A. (1990-1991). Competition, cooperation, satisfaction,
and the performance of complex tasks among Air Force cadets. Current Psychology:
Research & Reviews, 9(4), 347-354.
Quilici, J. L., & Mayer, R. E. (1996). Role of examples in how students learn to categorize
statistics word problems. Journal of Educational Psychology, 88(1), 144-161.
Ramsberger, P. F., Hopwood, D., Hargan, C. S., & Underhill, W. G. (1983), Evaluation of a
spatial data management system for basic skills education. Final Phase I Report for Period
7 October 1980 - 30 April 1983 (HumRRO FR-PRD-83-23). Alexandria, VA: Human
Resources Research Organization.
Randel, J. M., Morris, B. A., Wetzel, C. D., & Whitehill, B. V. (1992). The effectiveness of
games for educational purposes: A review of recent research. Simulation & Games, 23,
261-276.
Navigation Maps and Problem Solving: revised 5/18/05
135
Renkl, A., & Atkinson, R. K. (2003). Structuring the transition from example study to problem
solving in cognitive skill acquisition: A cognitive load perspective. Educational
Psychologist, 38(1), 13-22.
Renkl, A., Atkinson, R. K., Maier, U. H., & Staley, R. (2002). From example study to problem
solving: Smooth transitions help learning. The Journal of Experimental Education, 70(4),
293-315.
Resnick, H., & Sherer, M. (1994). Computerized games in the human services--An introduction.
In H. Resnick (Ed.), Electronic Tools for Social Work Practice and Education (pp. 5-16).
Bington, NY: The Haworth Press.
Ricci, K. E. (1994, Summer). The use of computer-based videogames in knowledge acquisition
and retention. Journal of Interactive Instruction Development, 7(1), 17-22.
Ricci, K. E., Salas, E., & Cannon-Bowers, J. A. (1996). Do computer-based games facilitate
knowledge acquisition and retention? Military Psychology, 8(4), 295-307.
Rieber, L. P. (1996). Seriously considering play: Designing interactive learning environments
based on the blending of microworlds, simulations, and games. Educational Technology
Research and Development, 44(2), 43-58.
Rieber, L. P., & Matzko, M. J. (Jan/Feb 2001). Serious design for serious play in physics.
Educational Technology, 41(1), 14-24.
Rieber, L. P., Smith, L., & Noah, D. (1998, November/December). The value of serious play.
Educational Technology, 38(6), 29-37.
Rosenorn, T., & Kofoed, L. B. (1998). Reflection in learning processes through
simulation/gaming. Simulation & Gaming, 29(4), 432-440.
Navigation Maps and Problem Solving: revised 5/18/05
136
Ruben, B. D. (1999, December). Simulations, games, and experience-based learning: The quest
for a new paradigm for teaching and learning. Simulations & Gaming, 30, 4, 498-505.
Ruddle, R. A., Howes, A., Payne, S. J., & Jones, D. M. (2000). The effects of hyperlinks on
navigation in virtual environments. International Journal of Human-Computer Studies,
53, 551-581.
Ruiz-Primo, M. A., Schultz, S. E., and Shavelson, R. J. (1997). Knowledge map-based
assessment in science: Two exploratory studies (CSE Tech. Rep. No. 436). Los Angeles,
University if California, Center for Research on Evaluation, Standards, and Student
Testing (CRESST).
Salas, E., Bowers, C. A., & Rhodenizer, L. (1998). It is not how much you have but how you use
it: Toward a rational use of simulation to support aviation training. The International
Journal of Aviation Psychology, 8(3), 197-208.
Salomon, G. (1983). The differential investment of mental effort in learning from different
sources. Educational Psychology, 18(1), 42-50.
Santos, J. (2002, Winter). Developing and implementing an Internet-based financial system
simulation game. The Journal of Economic Education, 33(1), 31-40.
Schau, C. & Mattern, N. (1997). Use of map techniques in teaching applied statistics courses.
American statistician, 51, 171-175.
Schraw, G. (1998). Processing and recall differences among seductive details. Journal of
Educational Psychology, 90(1), 3-12.
Shyu, H.-y., & Brown, S. W. (1995). Learner-control: The effects of learning a procedural task
during computer-based videodisc instruction. International Journal of Instructional
Media, 22(3), 217-230.
Navigation Maps and Problem Solving: revised 5/18/05
137
Spiker, V. A., & Nullmeyer, R. T. (n.d.). Benefits and limitations of simulation-based mission
planning and rehearsal. Unpublished manuscript.
Stewart, K. M. (1997, Spring). Beyond entertainment: Using interactive games in web-based
instruction. Journal of Instructional Delivery, 11(2), 18-20.
Story, N., & Sullivan, H. J. (1986, November/December). Factors that influence continuing
motivation. Journal of Educational Research, 80(2), 86-92.
Sweller, J., & Chandler, P. (1994). Why some material is difficult to learn. Cognition and
Instruction, 12, 185-233.
Tarmizi, R. A., & Sweller, J. (1988). Guidance during mathematical problem solving. Journal of
Educational Psychology, 80(4), 424-436.
Tennyson, r. D., & Breuer, K. (2002). Improving problem solving and creativity through use of
complex-dynamic simulations. Computers in Human Behavior, 18(6), 650-668.
Thiagarajan, S. (1998, Sept/October). The myths and realities of simulations in performance
technology. Educational Technology, 38(4), 35-41.
Thomas, P., & Macredie, R. (1994). Games and the design of human-computer interfaces.
Educational Technology, 31, 134-142.
Thompson, L. F., Meriac, J. P., & Cope, J. G. (2002, Summer). Motivating online performance:
The influence of goal setting and Internet self-efficacy. Social Science Computer Review,
20(2), 149-160.
Thorndyke, P. W., & Hayes-Roth, B. (1982). Differences in spatial knowledge acquired from
maps and navigation. Cognitive Psychology, 14, 560-589.
Tuovinen, J. E., & Sweller, J. (1999). A comparison of cognitive load associated with discovery
learning and worked examples. Journal of Educational Psychology, 91(2), 334-341.
Navigation Maps and Problem Solving: revised 5/18/05
138
van Merrienboer, J. J. G., Clark, R. E., & de Croock, M. B. M. (2002). Blueprints for complex
learning: The 4C/ID-model. Educational Technology Research & Development, 50(2),
39-64.
van Merrienboer, J. J. G., Kirschner, P. A., & Kester, L. (2003). Taking a load off a learner’s
mind: Instructional design for complex learning. Educational Psychologist, 38(1), 5-13.
Wainess, R., & O’Neil, H. F. Jr. (2003, August). Feasibility study: Video game research
platform. Manuscript: University of Southern California.
Washbush, J., & Gosen, J. (2001, September). An exploration of game-derived learning in total
enterprise simulations. Simulation & Gaming, 32(3), 281-296.
West, D. C., Pomeroy, J. R., Park, J. K., Gerstenberger, E. A., Sandoval, J. (2000). Critical
thinking in graduate medical education. Journal of the American Medical Association,
284(9), 1105-1110.
Westbrook, J. I., & Braithwaite, J. (2001). The Health Care Game: An evaluation of a heuristic,
web-based simulation. Journal of Interactive Learning Research, 12(1), 89-104.
Westerman, S. J. (1997). Individual differences in the use of command line and menu computer
interfaces. International Journal of Human-Computer Interaction, 9(2), 183-198.
White, R. W. (1959). Motivation reconsidered: The concept of competence. Psychological
Review, 66, 297-333.
Wiedenbeck, S., & Davis, S. (1997). The influence of interaction style and experience on user
perceptions of software packages. International Journal of Human-Computer Studies, 46,
563-588.
Navigation Maps and Problem Solving: revised 5/18/05
139
Wolfe, J. (1997, December). The effectiveness of business games in strategic management
course work [Electronic Version]. Simulation & Gaming Special Issue: Teaching
Strategic Management, 28(4), 360-376.
Wolfe, J., & Roge, J. N. (1997, December). Computerized general management games as
strategic management learning environments [Electronic Version]. Simulation & Gaming
Special Issue: Teaching Strategic Management, 28(4), 423-441.
Yair, Y., Mintz, R., & Litvak, S. (2001). 3D-virtual reality in science education: An implication
for astronomy teaching. Journal of Computers in Mathematics and Science Teaching,
20(3), 293-305.
Yeung, A. S., Jin, P., & Sweller, J. (1997). Cognitive load and learner expertise: Split-attention
and redundancy effects in reading with explanatory notes. Contemporary Educational
Psychology, 23, 1-21.
Yu, F.-Y. (2001). Competition within computer-assisted cooperative learning environments:
Cognitive, affective, and social outcomes. Journal of Educational Computing Research,
24(2), 99-117.
Yung, A. S. (1999). Cognitive load and learner expertise: Split attention and redundancy effects
in reading comprehension tasks with vocabulary definitions. Journal of Educational
Media, 24(2), 87-102.
Zimmerman, B. J. (1994). Dimensions of academic self-regulation: A conceptual framework for
education. In D. H. Schunk, & B. J. Zimmerman (Eds.), Self-regulation of learning and
performance (pp. 3-21). Hillsdale, NJ: Erlbaum.
Zimmerman, B. J. (2000). Self-efficacy. An essential motive to learn. Contemporary Educational
Psychology, 25(1), 82-91.
Navigation Maps and Problem Solving: revised 5/18/05
140
APPENDIX A
Study Examining Problem-Solving
With Video Games!!!
Participants Wanted!
College students or graduates, 18 years or older,
with no experience playing the video game
SafeCracker®, needed to participate in a University
of Southern California Research Study on ProblemSolving Tasks Using A Computer-Based Video
Game.
Study takes approximately 90 minutes.
Participants are paid $15.
If you are interested, please contact:
Richard Wainess
Ph. D. Student
University of Southern California
wainess@usc.edu
Navigation Maps and Problem Solving: revised 5/18/05
141
APPENDIX B
Demographic Information
User ID: _______________________
1.
Gender:
 Male
2.
Age:
_____ Years ______ Months
3.
Average amount per week you play video games (please check one)
 None
4.
 1-2 hours
 Female
 3-6 hours
 more than 6 hours
On a scale from 1 to 5, rate how much you like playing each game type. One
is lowest and five is highest, If you’re unsure or if you don’t know a term,
enter a 0 (zero) for the game. Be sure to enter a number for every game type.
a. Puzzle Games
_____ (1 to 5; or zero for “don’t know”)
b. RTS Games
_____
c. FPS Games
_____
d. Strategy Games
_____
e. Role Playing Games
_____
f. Arcade Games
_____
g. PC Games
_____
h. Console Games
_____
Navigation Maps and Problem Solving: revised 5/18/05
APPENDIX C
Task Completion Form (task 1)
User ID: _______________________
Mark the box for each safe you opened during each game.
FIRST GAME: (reception, small showroom, technical design room)
Small Showroom
 Blue Safe
 Brown Safe
 Gray Safe
Technical Design Room
 Blue Safe
 Strongbox (file cabinet in storeroom)
142
Navigation Maps and Problem Solving: revised 5/18/05
143
Task Completion Form (task 2)
User ID: _______________________
Mark the box for each safe you opened during each game.
SECOND GAME: (Chief engineers room, constructor office, technical design
room)
Constructor’s Office
 Liberty Safe
Chief Engineer’s Office
 Green Safe
 Archive
Technical Design Room
 Blue Safe
 Strongbox (file cabinet in storeroom)
Navigation Maps and Problem Solving: revised 5/18/05
144
APPENDIX D
Self-Regulation Questionnaire
Name (please print): _________________________________________________________________
Directions: A number of statements which people have used to describe themselves are given
below. Read each statement and indicate how you generally think or feel on learning tasks by
marking your answer sheet. There are no right or wrong answers. Do not spend too much time on
any one statement. Remember, give the answer that seems to describe how you generally think
or feel.
Almost
Almost
Never
Sometimes
Often
Always
1.
I determine how to solve a task before I begin.
1
2
3
4
2.
I check how well I am doing when I solve a
1
2
3
4
task.
3.
I work hard to do well even if I don't like a task.
1
2
3
4
4.
I believe I will receive an excellent grade in
1
2
3
4
courses.
5.
I carefully plan my course of action.
1
2
3
4
6.
I ask myself questions to stay on track as I do a
1
2
3
4
1
2
3
4
1
2
3
4
task.
7.
I put forth my best effort on tasks.
8.
I’m certain I can understand the most difficult
material presented in the readings for courses.
Navigation Maps and Problem Solving: revised 5/18/05
Almost
9.
I try to understand tasks before I attempt to
145
Almost
Never
Sometimes
Often
Always
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
solve them.
10.
I check my work while I am doing it.
11.
I work as hard as possible on tasks.
12.
I’m confident I can understand the basic
concepts taught in courses.
13.
I try to understand the goal of a task before I
attempt to answer.
14.
I almost always know how much of a task I
have to complete.
15.
I am willing to do extra work on tasks to
improve my knowledge.
16.
I’m confident I can understand the most
complex material presented by the teacher in
courses.
17.
I figure out my goals and what I need to do to
accomplish them.
18.
I judge the correctness of my work.
Navigation Maps and Problem Solving: revised 5/18/05
Almost
19.
I concentrate as hard as I can when doing a task.
20.
I’m confident I can do an excellent job on the
146
Almost
Never
Sometimes
Often
Always
1
2
3
4
1
2
3
4
1
2
3
4
assignments and tests in courses.
21.
I imagine the parts of a task I have to complete.
22.
I correct my errors.
1
2
3
4
23.
I work hard on a task even if it does not count.
1
2
3
4
24.
I expect to do well in this course.
1
2
3
4
25.
I make sure I understand just what has to be
1
2
3
4
1
2
3
4
done and how to do it.
26.
I check my accuracy as I progress through a
task.
27.
A task is useful to check my knowledge.
1
2
3
4
28.
I’m certain I can master the skills being taught
1
2
3
4
in courses.
29.
I try to determine what the task requires.
1
2
3
4
30.
I ask myself, how well am I doing, as I proceed
1
2
3
4
through tasks.
Navigation Maps and Problem Solving: revised 5/18/05
Almost
147
Almost
Never
Sometimes
Often
Always
31.
Practice makes perfect.
1
2
3
4
32.
Considering the difficulty of courses, teachers,
1
2
3
4
and my skills, I think I will do well courses.
Copyright © 1995, 1997, 1998, 2000 by Harold F. O’Neil, Jr.
Navigation Maps and Problem Solving: revised 5/18/05
APPENDIX E
SafeCracker® Expert Map
148
Navigation Maps and Problem Solving: revised 5/18/05
149
APPENDIX F
Knowledge Map Specifications
General Domain
This Software
Specification
Scenario
Create a knowledge map on the content understanding of
SafeCracker, a computer puzzle-solving game.
Participants
College students or graduate students. Each works on his/her own
knowledge map about SafeCracker, after both the first time and
the second time of playing the game.
Knowledge map
Fifteen predefined key concepts identified in the content of
concepts/nodes
SafeCracker, by experts of the game and knowledge map
professionals. The fifteen predefined concepts are: book, catalog,
clue, code, combination, compass, desk, direction, floor plan,
key, room, safe, searching, trial-and-error, and tool.
Knowledge map
Seven predefined important links of relationships identified in the
links
content of SafeCracker by experts of the game and knowledge
map professionals. The seven predefined links are: causes,
contains, leads to, part of, prior to, requires, and used for.
Knowledge map
SafeCracker is a computer puzzle-solving game. There are over
domain/content:
50 rooms with about 30 safes; each safe is a puzzle to solve. To
SafeCracker
solve the puzzles, players need to find out clues and tools hidden
Navigation Maps and Problem Solving: revised 5/18/05
in the rooms, deliberate and reason out the logic and sequence,
try to apply what they have found.
Training of the
All students will go through the same training session.
computer knowledge
The training included the following elements:
mapping system
• How to construct a knowledge map using the computer
mapping system
• How to play SafeCracker, which is the target domain/content
of the programmed knowledge mapper.
Type of knowledge
Problem solving
to be learned
Three problem
solving measures
1. Knowledge map used to measure content understanding and
structure, including (a) semantic content score; (b) the
number of concepts; and (c) the number of links
2. Domain specific problem-solving strategy questionnaire,
including questions to measure problem-solving retention
and transfer
3. Trait self-regulation questionnaire used to measures the four
elements of trait self-regulation: planning, self-checking,
self-efficacy, and effort
150
Navigation Maps and Problem Solving: revised 5/18/05
151
APPENDIX G
LEARNING SAFECRACKER®
First trial will use reception, small showroom, and technical design room
(Rooms 1, 2, and 27)
Second trial will use constructor office, chief engineer room, and technical design room
(Rooms 5, 6, and 27)
In this study, you will be asked to accomplish a series of tasks. The tasks will be to locate
and open various safes in various rooms in a mansion. In order to open some of these safes, you
will need to find certain items. You will be told which rooms to visit. Those rooms contain all
the items needed and all the safes you will need to open. You do not need to spend time in any
other rooms. Even though the mansion has two floors, all the rooms you will visit are on the first
floor. Do not go to the second floor.
Your goal is to open all the safes in the rooms you are given. For each room, you will be
told the room’s name (e.g., the Small Showroom).
Together, we will walk through the steps needed to find and enter the mansion. Then, we
will walk through searching the first room and opening one safe. After that, you will given the
names of several rooms and will be required to find the rooms and open the safes. From this
point on, DON’T DO ANYTHING UNLESS I EXPLICITLY TELL YOU TO with phrases like
“OKAY, DO IT NOW” or “GO AHEAD AND DO IT.” Does everyone understand?
GETTING INTO THE MANSION
Once I tell you to click the “new” button to begin a new game, you’ll see the game’s
main interface screen and a phone. You’ll be in a phone booth facing the phone. The phone will
be ringing, but you won’t hear it because the sound is turned off on your computer. As you move
Navigation Maps and Problem Solving: revised 5/18/05
152
your cursor to the phone’s handpiece, a hand symbol will appear on your cursor. This means you
can click on the phone piece to remove it from its hook. Anytime you see a hand symbol it
means you can click on something. Once you remove the hand piece, a voice will begin
speaking. Unfortunately, you won’t be able to hear it because, as already mentioned, the sound is
turned off on your computer. So right now, I’m going to tell you the most important information
you would have heard. And I need you to write it down on your scratch paper as I say it to you.
That information is a four-digit code you’re going to need to use in order to enter the mansion.
That code is 1923. Write that down now; 1923.
Once you remove the hand piece, wait about five seconds, move your cursor back to the
hook. When you see the hand symbol, click to hang up the phone, then wait for further
instructions. To repeat, you will click the phone piece, wait about five seconds, and click the
handle to hang up the phone. After that, you will do nothing. That includes not moving the
cursor. Do you have any questions? Okay, go ahead now and click NEW to start the game.
(Once everyone’s done listening to the message).
Move your cursor about 2 inches to the right of the hand piece, hold down the cursor, and
move your mouse left and right. Your view will pan left and right. The further you move left or
right, the faster the scene will pan. You can also move the cursor up or down to tilt up or down a
little. If you stop moving the mouse but continue to hold the mouse button down, the scene will
continue to pan. (Wait a few seconds). You can also use the left, right, up, and down cursor
arrows pan or tilt your view.
Now let’s exit the phone booth. Rotate until you see the sidewalk. When you let go of the
cursor, you should see the hand symbol. If you don’t, rotate 180 degrees and look at the sidewalk
going the other direction. Then click once to open the phone booth door and a second time to exit
Navigation Maps and Problem Solving: revised 5/18/05
153
the phone booth. Only click twice. After that, wait for instructions. Okay, go ahead and to that
now.
(Once everyone’s outside the mansion).
Now, I’m going to give you some instructions and it’s very important that you do
absolutely nothing until I tell you to. (beat) Across the street is a lit up, two story mansion. Do
you see how much of your sidewalk you can see? When I tell you to, you’re going to slowly
rotate to the right. You’ll stop when you can see as much of the mansion as you can, while still
seeing as much of your sidewalk as you currently see. Go ahead and do that, then wait for further
instructions.
Once again, I’m going to give you a series of instructions and it’s very important that you
do not do anything until I explicitly tell you to. (beat) Do you see those white stripes crossing the
street? That’s a crosswalk. You’re going to walk down your sidewalk, turn right and cross the
street. Then you’ll turn left and walk further down the other sidewalk. Then you’ll turn to the
right and face the mansion. (beat) More specifically, you’ll take two steps down your sidewalk.
You’ll turn to the right and take two steps to cross the street. You’ll turn to the left and take two
more steps to walk further down the other sidewalk. Then you’ll turn right and face the mansion.
Okay, go ahead and get to the mansion.
(Once everyone’s facing the gate).
You should be facing a gate, with the mansion in the background. See the lock on the
gate. Go ahead and click on it. You’ll need to see the hand symbol in order to click. (wait) Now,
notice the three tumblers on the Go ahead and click on the tumblers, and you’ll notice that all
three can be rotated. If you rotate them to the correct pattern, the lock will open. I’ll give you one
minute to try. Go ahead now and try to open the lock.
Navigation Maps and Problem Solving: revised 5/18/05
154
(After one minute).
If you haven’t opened the lock yet, set the three tumblers to music symbols and the lock
will open.
(Once all locks are opened).
Now go ahead and move to the front door. You’ll need to navigate a little bit around that
fountain that’s up ahead.
(Once everyone’s at the front door).
Notice how the front entrance has two doors. I want you do rotate so that you see the area
just to the left of the left door. Go ahead and start rotating. Notice that small gray box? Go ahead
and click on it. (wait) Now, using your mouse, click the four digits I had you write down earlier,
then click on the ENTER button on that keypad. Go ahead and do that now. Remember to you
the mouse to click on numbers, rather than using your keyboard.
(Once everyone’s gained access).
Now click once to open the first door. Now click again to open the second door and a
third time to enter the mansion.
(Once everyone’s inside the mansion).
Rotate left and right and you’ll notice you’re in front of a reception desk. The secretary’s
chair to your left is facing a computer. Rotate and you should be able to see the back of the
monitor. Go ahead and navigate around until you’re facing that computer (wait). Once you’re
there, click on the computer screen once and then wait. Go ahead.
(Once everyone’s at the computer).
Click once more on the computer screen. Then wait about five seconds.
Navigation Maps and Problem Solving: revised 5/18/05
155
(Once everyone is looking at minesweeper).
Now try to move your mouse to look around. Notice how nothing happens. This is
because you’re locked on to the computer screen. Whenever you’re locked onto something, you
can’t do anything else until you BACK AWAY from that object. To do that, click the BACK
button that’s on the right side of the screen. Go ahead and do that.
(Once everyone’s backed up).
Now, click on the blue cup to the left of the computer. A larger view of the cup appears
in a window at the bottom of the screen. You can grab the cup on that window and rotate it. Go
ahead and try that.
(After everyone’s rotated the cup).
To the left of the blue cup is a piece of paper. Go ahead and click on it. Notice the
diagrams. These might be important for opening a safe. You might want to write them down
later, when you start playing the game. If you move your cursor near the bottom of the paper, a
down arrow cursor appears, indicating that you can click to see more of the paper. Go ahead and
do that. You can also click to the right to see more of the paper. And click near the top to see the
top portion of the paper.
(Once everyone’s seen all parts of the paper).
Go ahead and click your back button. (Wait a couple seconds). Now rotate to the right
and find another piece of paper. Go ahead and click on that. You can move your cursor to the
bottom and click to see more. With this paper, you can’t click to see the right portion. (wait) Go
ahead and click the back button.
Navigation Maps and Problem Solving: revised 5/18/05
156
Rotate a little more to the right and you’ll see a third piece of paper. Go ahead and click
on that paper. You can click down on the paper and to the right to see more. This paper is filled
with diagrams that might be helpful for opening safes. (beat) Once you’ve seen the whole paper,
click the back button and then don’t do anything else.
(Wait until everyone’s seen both papers).
I don’t want you to click on anything else, but notice there are several books. Some
contain potentially useful information. If you rotated around, you’d see other items around the
desk that might be worth looking at, including more papers and more books. Go ahead and rotate
around.
(wait a few seconds)
Now I’d like you to face that computer screen you went to earlier. (wait). See that blue
safe in the background? I want you to navigate to it. Go ahead. (beat) When you get there, click
on the safe to lock on to it and don’t do anything else. BE SURE NOT TO CLICK ON
ANYTHING. (wait) You’ll know you’re locked onto the safe when the button on the right side
of the screen said BACK.
(Once everyone is near the safe).
Once again, wait until I tell you to before clicking on anything. (beat). The safe has three
red lights, three white dials, and a handle just below the middle dial. The safe will open when the
three dials are set to the correct numbers. (beat) The red lights are connected to the dials. The
left dial controls the top red light. The middle dial controls the middle red light. And the right
dial controls the bottom red light.
Go ahead and click the handle now and you’ll notice one of the lights remains a solid
green while the other two flash. If a light remains green, it means that its dial is set correctly.
Navigation Maps and Problem Solving: revised 5/18/05
157
You can keep clicking on the handle if you want. (pause) When you click, the middle light stays
solid green, so the middle dial must be set correctly. I’ll want you to adjust the dials until all the
lights stay green. Go ahead and do that now. And remember, the middle dial is correct so don’t
move it.
(Once everyone’s opened the safe).
Click on the piece of paper. Click it again to make it go away. Notice it’s been added to
your inventory on the bottom right window of the screen. Click on it in the inventory to open it
up again. Click on it once more to make it go away. (beat) Now, click on the coins to add those
to your inventory as well. Some items can be used to open safes; for example, keys might be
used to open a lock or coins might be inserted into a coin slot. (Wait a second). To use an item,
you click on the item to make it active. Right now, your coins are active. Notice your inventory
text? And notice that window to the left of the inventory that shows a 3D image of the coins?
Now, notice the vertical button between them that says “USE.” That’s the button you click in
order to use something. Go ahead and click it now. (wait). If you were at something that could
use the coins, they would have been used. But since you weren’t, you might be able to use them
later. (beat) Now, back away from the safe.
(Wait until everyone’s backed away from the safe).
Now click on the safe to try to lock on to it. Notice you can’t and a red error message
appears at the top of the screen. That message states that the safe has already been cracked.
ONCE YOU CLOSE A SAFE, YOU CAN NEVER OPEN IT AGAIN. SO BE SURE TO
COLLECT ALL ITEMS FROM A SAFE BEFORE BACKING AWAY.
Notice that area where the red warning sign appeared? It now says “Reception.” That’s
the name of the room you’re in. (beat) When it’s not displaying an error message, that’s the room
Navigation Maps and Problem Solving: revised 5/18/05
158
indicator window and it lets you know the name of the room you’re in. (pause). Also notice the
compass at the bottom right of the computer screen. You might find that compass helpful for
determining which direction you’re facing or moving.
(FOR MAP USERS, READ THE “SCRIPT FOR INTRODUCING MAP TO
PARTICPANTS”)
(FOR NON-MAP USERS, READ THE “SCRIPT FOR THE CONTROL GROUP ON HOW
TO NAVIGATE THE MANSION”)
(THEN, ANNOUNCE THE FIRST TASK AND THE ROOMS INVOLVED. FOR THE MAP
USERS, HAND OUT THE APPROPRIATE MAP.)
Navigation Maps and Problem Solving: revised 5/18/05
159
APPENDIX H
SCRIPT FOR INTRODUCING MAP TO PARTICIPANTS
(Hand out the training map, entitled “how to read the map.”)
This is a map of a portion of the first floor of the mansion. You will use a map similar to
this to help navigate to the various rooms. Currently, you’re in, the reception room, the large
room in the middle of the bottom portion of the map. The map shows some of the rooms on the
first floor with their related names. These will match the names of the rooms that contain the
safes you will be asked to open and the items that will help you to open the safes. The names
also match the names that appear in the name indicator on your interface, which you have
already been shown. You will not need to visit any other rooms, unless they are along a path you
take in order to get to a required room. In addition to the room names, the map also shows the
location of the doors in each room. If you need to, you are allowed to write on this map.
This map shows a portion of the bottom floor and includes text labels describing of the
most important map features. On the left side are four labels. The top label on the left contains
the words “room name” and points to the name of the room entitled “Big Showroom.” Take a
look and you’ll see that every room has a name. Those areas that do not have names are either
closets or bathrooms. For each of your two tasks, you will be told the names of the rooms you
must visit. As shown to you earlier, there is a room name indicator on the interface. You will use
this indicator to verify which room you are in.
The middle label on the left contains the word “stairs” and points to a block of black and
gray stripes. That pattern indicates stairs. Notice that there’s another set of stairs just to the right
and a small set of stairs connecting the two.
Navigation Maps and Problem Solving: revised 5/18/05
160
On the left side, near the bottom, is a label with the word “door.” Gaps or open spaces
between rooms indicate doors. Every room has at least one door and most have several doors.
On the left side, at the bottom is a label with the words “Main Entrance” and an arrow
pointing to the door you came through to enter the mansion. This is the only door on the map that
is not indicated using an opening or gap. Once you began moving around the reception room,
that door locks and cannot be opened. That’s why it is not indicated by an opening.
In the middle of the map is a label with the word “toilet.” The arrow points to a small
circle, which is the symbol for a toilet. There are other bathrooms in the mansion that have
toilets, but for some reason, the people who created this map chose to only show this toilet.
On the right side of the map are three labels. The one on the far right side of the map and
containing the words “points north” points to a symbol with a black circle, a spike pointing
upward, and two spike pointing downward. This is a typical map indicator that shows direction
for “North.” The spike to points upward is pointing “north.”
On right side, just below and to the left of the “points north” label is a label with the word
“closet.” As mentioned before, closest and bathrooms don’t have room names. The one
exception is the room with the toilet. That room’s name is W.C., which stands for “water closet.”
Water closet is a term used in England for bathroom.
The last label is at the bottom on the right side of the mansion and contains the word
“door.” The three arrows emanating from that label point to three more examples of doors.
The last part of the map to show you is the darkened rooms. In the map you’re looking at,
there are three darkened rooms. They are “reception,” the “small showroom,” and the “technical
design” room. Just above the technical design room is a dark label with the words “your task
takes place in the shaded rooms.” As already mentioned, you will be given two tasks. For each
Navigation Maps and Problem Solving: revised 5/18/05
161
task you will be given a map. Each map will have a different set of darkened rooms, indicating
the rooms you must visit in order to complete each task. While you are allowed to visit other
rooms, your time to complete each task is limited, so it is best to not waste time visiting
unnecessary rooms.
As a first step for each task, it is recommended that you examine the map to determine
the shortest or most efficient paths for getting from room to room, and return to the various
rooms. As an example, in the current map, since you’re already in the reception room, if you
wanted to go to the “Small Showroom,” you’d use the right door of the reception room to enter
the “Small Showroom.” If you wanted to go from the small showroom to the technical design
room, there are no doors leading directly from one room to the other; there are no openings.
Instead, you’d need to go first to your right and enter the “Designer’s room.” Then you move up
the left side of that room and through a door that leads into the “Technical Design room.” To
return to the “Small Showroom,” you’d simply reverse your path.
Once you have a plan for how you will navigate to and from rooms, then you would
begin moving around, collecting items and attempting to open safes. REMEMBER, IT IS VERY
IMPORTANT THAT YOU LOOK AT ALL THE ITEMS IN ROOMS, TO FIND CLUES
THAT MIGHT HELP OPEN SAFES. Not everything gets added to your inventory. You may
need to write things down.
Do you have any questions?
Navigation Maps and Problem Solving: revised 5/18/05
162
APPENDIX I
SCRIPT FOR THE CONTROL GROUP ON HOW TO NAVIGATE THE MANSION
For each of your two tasks, you will need to navigate to three rooms and return to the
rooms by retracing your path. For each task, you will be told the name of the rooms you will
need to visit. As just shown, the interface includes a window that displays the name of the room
you’re in. Be sure to keep track of your room location. Because you will need to find your way
and than find your way back, use whatever methods you think will help to keep track of where
you’ve been and the path you’ve taken. NOTE: YOU WILL NEED TO GO THROUGH
OTHER, NON-TASK RELATED ROOMS TO GET TO YOUR ROOMS.
REMEMBER, IT IS VERY IMPORTANT THAT YOU LOOK AT ALL THE ITEMS IN
ROOMS, TO FIND CLUES THAT MIGHT HELP OPEN SAFES. Not everything gets added to
your inventory. You may need to write things down.
Do you have any questions?