The Effect of Navigation Maps on Problem Solving
Tasks Instantiated in a Computer-Based Video Game
Committee Members: Dr. Harold O’Neil (Chair)
Dr. Richard Clark
Dr. Edward Kazlauskas
Dr. Janice Schafrik
Dr. Yanis Yortsos (Outside Member)
Richard Wainess
Dissertation Proposal Presented to the
Faculty of the Graduate School
University of Southern California
April 27, 2004
Richard Wainess Dissertation Proposal v.3
Presented 04/27/04
Slide 1 of 16
Research Questions
• Will the problem solving performance of participants who
use a navigation map in a 3-D, occluded, computer-based
video game (i.e., SafeCracker®) be better than the problem
solving performance of those who do not use the map (the
control group)?
• Will the continued motivation of participants who use a
navigation map in a 3-D, occluded, computer-based video
game (i.e., SafeCracker®) be greater than the continued
motivation of those who do not use the navigation map
(the control group)?
Richard Wainess Dissertation Proposal v.3
Presented 04/27/04
Slide 2 of 16
Cognitive Load Theory
• Auditory/Verbal and Visual/Spatial Channels/Memory
(Baddeley,1986; Mayer & Moreno, 2003)
• Limited Working Memory (Brunken et al., 2003)
– 7 +/- 2 (Miller, 1956)
– Possibly only 2 or 3 novel elements (Paas et al., 2003)
• Unlimited Long-Term Memory (Mousavi et al., 1995)
• Cognitive Load = mental capacity imposed on working memory
(Sweller & Chandler, 1994)
– Controlled by schema development & Automation
Mayer & Moreno, 2001
Audio/Visual Info
Words
Pictures
Sensory Memory
Ears
Eyes
Working Memory
Selecting
Words
Sounds
Selecting
Images
Images
Richard Wainess Dissertation Proposal v.3
Organizing
Words
Verbal Model
Organizing
Pictorial Model
Images
Presented 04/27/04
Integrating
Prior
Knowledge
Slide 3 of 16
Cognitive Load Theory (cont’d)
• Intrinsic Cognitive Load: The process of integrating new
information with existing knowledge: e.g., working memory, long
term memory, metacognition (Brunken et al., 2003; Paas et al., 2003).
• Germane Cognitive Load: The cognitive load required to process
intrinsic load (Renkl & Atkinson, 2003).
– Imposed by the environmental requirements related to learning (e.g.,
instruction, reading, searching, problem solving, interface elements)
• Extraneous Cognitive Load: Imposed by an unnecessary stimuli:
e.g., interface artwork, extraneous sounds (Brunken et al. 2003).
– Seductive Details (Mayer et al., 2001; Schraw, 1998).
• Learner Control: pacing & navigation (Barab et al., 1999; Cutmore et al.,
2000)
– Mixed or Negative Results (Bernard et al., 2003; Niemiec et al., 1996)
Richard Wainess Dissertation Proposal v.3
Presented 04/27/04
Slide 4 of 16
Games and Simulations
• Games: rules, constraints/privileges, imaginative, linear
(Gredler, 1996)
• Simulation-Games: combination of games and simulations
(Gredler, 1996)
• Motivation in games: fantasy, control & manipulation,
challenge & complexity, curiosity, competition, feedback, fun
Richard Wainess Dissertation Proposal v.3
Presented 04/27/04
Slide 5 of 16
Games and Simulations
Outcomes with Games and Simulations
• Positive outcomes:
– Numerous knowledge outcomes attributed to games and simulations
• Warning about anecdotal and descriptive evaluations (Leemkuil et al., 2003;
Wolfe, 1997)
– Generalizable skills outcomes
(Day et al. 2001; Green & Bavelier, 2003; Greenfield et al., 1994)
• Negative or null outcomes:
– Reviews and meta-analyses cite mixed or negative reviews
(Dekkers & Donatti, 1981; Druckman, 1995)
– Positive attitude toward games doesn’t necessarily equal learning
(Brougere, 1999; Salas et al., 1998; Salomon, 1984)
• Outcomes related to Instructional Design, not games/simulations
(de Jong & van Joolingen, 1998; Garris et al., 2002; Gredler, 1996; Leemkuil et al., 2003;
Thiagarajan, 1998)
Richard Wainess Dissertation Proposal v.3
Presented 04/27/04
Slide 6 of 16
Scaffolding
(including Graphical Scaffolding)
• Instructional methods
– Should keep cognitive load low (Clark, 2003)
– External methods which replace internal processes (Clark, 2001)
• Scaffolding is an instructional method
• Scaffolding provides support during learning
(Allen, 1997; Chalmers, 2003; van Merrienboer et al., 2002, 2003)
• Graphical Scaffolding
– Includes maps and menus as advance organizers (Jones et al., 1995)
– Maps supported by researchers as visual aids and organizers
(Benbasat & Todd, 1993; Chou & Lin, 1998; Ruddle et al., 1999)
– Should be used for visual tasks (Mayer et al., 2002)
Richard Wainess Dissertation Proposal v.3
Presented 04/27/04
Slide 7 of 16
Navigation Maps
• Navigation maps effective in 3-D, occluded, environments with
simple problem solving tasks (Galimberti, 2001)
Not yet examined
Navigation in 3D, occluded, environments
with complex problem solving tasks
Richard Wainess Dissertation Proposal v.3
Presented 04/27/04
Slide 8 of 16
Navigation Map
Floor Plan of Mansion’s First Floor from SafeCracker®
Richard Wainess Dissertation Proposal v.3
Presented 04/27/04
Slide 9 of 16
Research Hypotheses
Problem Solving
– Hypothesis 1: Navigation maps will produce a significant increase in
content understanding compared to the control group.
– Hypothesis 2: Navigation maps will produce a significant increase in
problem solving strategy retention compared to the control group.
– Hypothesis 3: Navigation maps will produce a significant increase in
problem solving strategy transfer compared to the control group.
– Hypothesis 4: There will be no significant difference in self-regulation
between the navigation map group and the control group. However, it is
expected that higher levels of self-regulation will be associated with better
performance.
Motivation
– Hypothesis 5: Navigation maps will produce a significantly greater
amount of optional continued game play compared to the control group.
Richard Wainess Dissertation Proposal v.3
Presented 04/27/04
Slide 10 of 16
Method
• Design
– Experimental: Random Assign to Treatment or Control Group
– Treatment receives navigation map; Control Group doesn’t
– Segregated group sessions: all Treatment or all Control
• Participants
– 60 males & females, ages 18-25, undergraduates at USC
– No prior experience playing SafeCracker®
• Procedure (90 minutes)
–
–
–
–
–
Demographic Information & Self-Regulation Questionnaire
Introduced to Knowledge Map
Handout Navigation Map to treatment group only
Introduced to the game (interface, opening locks, finding objects)
Pretest: Knowledge map
Richard Wainess Dissertation Proposal v.3
Presented 04/27/04
Slide 11 of 16
Method (cont’d)
• Procedure (cont’d)
– Play game (first set of rooms)
– Intermediate Test
• Knowledge Map; Retention/Transfer Questions; Task Check-List
– Play Game (second set of rooms)
– Post Test:
• Knowledge Map; Retention/Transfer Questions, Task Check-List
– Debriefing
– Optional playing time (up to 30 minutes)
• Measures
– Problem Solving (O’Neil, 1999)
• Content Understanding
• Problem Solving Strategies
• Self Regulation
– Motivation
• Verbalized desire to continue playing
• Amount of free play minutes
Richard Wainess Dissertation Proposal v.3
Presented 04/27/04
Slide 12 of 16
Sample SafeCracker® Knowledge Map
desk
brochure
contains
contains
contains
results from
contains
room
contains
key
part of
clue
causes
safe
contains results from
results from
contains
causes
part of
crack
uses
books
used for
tool
Richard Wainess Dissertation Proposal v.3
used for
causes
direction
used for
Presented 04/27/04
map
Slide 13 of 16
Instrument: Problem Solving
Retention & Transfer Questions
Retention questions:
1. List how you opened the safe in the first room.
2. List how you opened the safe in the second room.
Transfer questions:
1. List some ways to improve the way you solved
opening the safe in room 1.
2. List some ways to improve the way you solved
opening the safe in room 2.
3. List some ways to improve the way you navigated
from room 1 to room 2.
Richard Wainess Dissertation Proposal v.3
Presented 04/27/04
Slide 14 of 16
Instrument: Self-Regulation
Self-Regulation Questionnaire
• Based on O’Neil (1999) Problem Solving Model
• Trait self-regulation questionnaire (O’Neil & Herl, 1998).
• 32 Questions: 8 each of four measures
–
–
–
–
planning
self-checking/monitoring
self-efficacy
effort
Richard Wainess Dissertation Proposal v.3
Presented 04/27/04
Slide 15 of 16
Data Analysis
•
Descriptive statistics: Means, Standard Deviation, etc., for all measures
•
T-Tests for the following
– Hypothesis 1: Navigation maps will produce a significant increase in content
understanding compared to the control group.
– Hypothesis 2: Navigation maps will produce a significant increase in problem
solving strategy retention compared to the control group.
– Hypothesis 3: Navigation maps will produce a significant increase in problem
solving strategy transfer compared to the control group.
– Hypothesis 5: Navigation maps will produce a significantly greater amount of
optional continued game play compared to the control group.
•
Pearson’s Correlation: Effect of the four self-regulation variables (planning,
self-checking/monitoring, self-efficacy, and effort) on content understanding
and problem solving strategies.
– Hypothesis 4: There will be no significant difference in self-regulation between the
navigation map group and the control group. However, it is expected that higher
levels of self-regulation will be associated with better performance.
Richard Wainess Dissertation Proposal v.3
Presented 04/27/04
Slide 16 of 16
BACK UP
Richard Wainess Dissertation Proposal v.3
Presented 04/27/04
Slide 17
Problem Solving Assessment
O’Neil (1999)
Problem
Solving Model
Problem
Solving
Content
Understanding
Problem-Solving
Strategies
Self-Regulation
Metacognition
Planning
Domain
Specific
SelfMonitoring
Motivation
Effort
SelfEfficacy
Domain
Independent
Richard Wainess Dissertation Proposal v.3
Presented 04/27/04
Slide 18
Navigation Maps
• Navigation: Tracking one’s position in an environment to arrive
at a destination (Cutmore et al., 2000)
• Occlusion: when a path is blocked visually (Cutmore et al., 2000)
• Navigation maps effective for occluded 3-D navigation
(Cutmore et al. 2000; Dempsey, 2002)
• Navigation maps effective in 2-D environments with complex
problem solving tasks (Baylor, 2001; Chou & Lin, 1998; Chou et al., 2000)
• Navigation maps effective in 3-D occluded environments with
simple problem solving tasks (Galimberti, 2001)
Not yet examined
Navigation in 3D, occluded, environments
with complex problem solving tasks
Richard Wainess Dissertation Proposal v.3
Presented 04/27/04
Slide 19
Instrument: Content Understanding
Knowledge Mapping Software
Richard Wainess Dissertation Proposal v.3
Presented 04/27/04
Slide 20
Problem Solving Assessment
• Based on O’Neil’s (1999) Problem Solving Model
– Requires content understanding, problem solving strategies, selfregulation
• Transfer questions are alternative to transfer tasks (Moreno & Mayer, 1998)
• Declarative knowledge measured by retention (Day et al., 2001)
• Knowledge map: concepts and links (Schau & Mattern, 1997)
– Reflects the organization of knowledge (Day et al., 2001)
– Reliable and efficient measure of Content Understanding
(Herl et al., 1999; O’Neil, 1999; Ruiz-Primo et al., 1997; Schacter et al., 1999)
Predictive of both retention and transfer (Day et al., 2001)
–
– Reliable measure of Problem Solving Strategies (Baker & Mayer, 1999)
Richard Wainess Dissertation Proposal v.3
Presented 04/27/04
Slide 21
Problem Solving Assessment (cont’d)
• Measuring Problem Solving Strategies
– Domain general and specific (Alexander, 1992; Bruning et al., 1999)
– Knowledge Mapping (Baker & Mayer, 1999)
– Problem solving questions
• Positively correlated with retention and transfer (Mayer & Baker, 1998)
• Measuring Self-Regulation
– Includes Metacognition & Self-efficacy (O’Neil, 1999)
• Metacognition: planning & self-checking (Pintrich & DeGroot, 1990)
• Self-efficacy: mental effort and self-efficacy (Zimmerman, 1994, 2000)
– Trait self-regulation questionnaire (Hong & O’Neil, 2001)
• Includes planning, self-checking, mental effort, and self-efficacy
• 32 questions: 8 for each of the four sub-categories
Richard Wainess Dissertation Proposal v.3
Presented 04/27/04
Slide 22