UNIVERSITY PARK INSTITUTIONAL REVIEW BOARD APPLICATION
SECTION II
TEMPLATE REV. 4/2003
Original Date: June 8, 2004
Revision Date: July 22, 2004
TITLE OF PROJECT:
The Effect of Navigation Maps on Problem Solving Tasks Instantiated in a Computer-Based Video
Game
PURPOSE OF THE STUDY AND BACKGROUND
1. Specific Aims and Purpose of the Study:
Our programmatic research is directed towards answering the following questions. (1) Will the
problem solving performance within a video game (i.e., SafeCracker) for participants who use a
navigation map be greater than for those participants who do not use the navigation map? (2)
Will continued motivation be greater for participants who use the navigation map than for those
who do not use the navigation map?
The study will primarily focus on the impact of the navigation map on problem solving. The
problem solving assessment is comprised of three components: content understanding, problemsolving strategies, and self-regulation. Content understanding (domain knowledge) is measured
by a concept map. Problem-solving strategies are measured by domain specific problem-solving
questions. Self-regulation is measured by a trait thinking questionnaire, which assesses
planning, self-checking, effort, and self-efficacy. The concept map and the self-regulation
questionnaire will be computer-based.
The entire session will take one 90 minute period, plus optional free play time of up to 30 minutes.
The procedure for the study is as follows: (a) 3 minutes for introduction, (b) 10 minutes for the
self-regulation questionnaire and demographic questionnaire, (c) 10 minutes for instruction of
knowledge mapping, (d) 10 minutes for game introduction, (e) 15 minutes for first game playing,
(f) 7 minutes for the first knowledge map drawing (intermediate), (g) 4 minutes for the first
problem-solving strategy questions (intermediate), (h) 1 minute for task completion questionnaire,
(i) 15 minutes for the second game-playing (j) 7 minutes for the second knowledge map drawing
(post), (k) 4 minutes for the second problem-solving strategy questions (post), (l) 1 minute for task
completion questionnaire, (5) 3 minutes for follow up, (n) up to 30 minutes of free optional play
time.
The self-regulation questionnaire contains 32 items with 8 items each representing each of four
trait factors: planning, self-checking/monitoring, self-efficacy, and effort. The knowledge map is
a graphically-based computer program containing lists of concepts (such as “safe” or “room”) and
links to connect the concepts (such as “leads to” or “requires”). Participants select concepts and
links to diagrammatically demonstrate their understanding of how the game works.
2. Background and Significance:
Research into the effectiveness of games and simulations as educational media has been met with
mixed reviews (de Jong & van Joolingen, 1998; Garris, Ahlers, & Driskell, 2002). It has been
suggested that the lack of consensus can be attributed to weaknesses in instructional strategies
embedded in the media and issues related to cognitive load (Chalmers, 2003; Cutmore, Hine,
Maberly, Langford, & Hawgood, 2000; Lee, 1999; Thiagarajan, 1998; Wolfe, 1997). A major
University of Southern California University Park IRB
Page 1 of 6
instructional issue in learning by doing within simulated environments concerns the proper type
of guidance, that is, how best to create cognitive apprenticeship (Mayer, Mautone, & Prothero,
2002).
A virtual environment poses a number of potential issues with regards to learning. Problemsolving within a virtual environment involves not only the cognitive load associated with the tobe-learned material (referred to as intrinsic cognitive load: Paas, Tuovinen, Tabbers, Van Gerven,
2003), it also includes cognitive load related to the visual nature of the environment (referred to as
extraneous cognitive load: Brunken, Plass, & Leutner; Harp & Mayer, 1998), as well as
navigating within the environment—either germane cognitive load or extraneous cognitive load,
depending on the relationship of the navigation to the learning task (Renkl, & Atkinson, 2003).
An important goal of instructional design within these immersive environments involves
determining methods for reducing the extraneous cognitive load and/or germane cognitive load,
thereby providing more working memory capacity for intrinsic cognitive load (Brunken et al.,
2003). This study will examine the reduction of cognitive load, by providing graphical scaffolding
in the form of a navigation map, to determine if this can result in better performance outcomes as
reflected in retention and transfer (Paas et al., 2003) in a game environment.
3. Progress Report/Preliminary Studies:
N/A
CHARACTERISTICS OF THE STUDY SUBJECT POPULATION
4. Target Accrual:
Three to four participants for the pilot study and 68 participants for the main study.
5. Age Range of Adult Subjects:
18 or older
6. Age Range of Minor Subjects:
N/A
7. Gender of the Subjects:
Male/Female
8. Racial and Ethnic Origin:
Non-specific
9. Inclusion of Special Classes or Vulnerable Subjects:
None
10. Inclusion Criteria:
The subject must be a college student, a graduate student, or a graduate.
11. Exclusion Criteria:
Anyone who is not a college student, or college graduate. Anyone who has played the video game
SafeCracker, prior to participation in the study.
METHODS AND PROCEDURES
12. Research Design and Methods Applied to Human Subjects:
Two studies will be conducted: a pilot study with between three and four participants, to evaluate
and fine tune the testing procedures, and the main study, which will be comprised of 68
IRB Application Section II (Rev. 4/2003) – Protocol Detail
Page 2 of 6
participants. Participants will be tested individually via computer systems which contain the game
and the concept mapping software, and the questionnaire. The demographics questionnaire, the
task completion sheet, and the problem-solving strategy questionnaire will be paper and pen based.
Subjects will also be given paper and pencil for note taking during game play.
The entire session will take one 90 minute period, plus optional free play time of up to 30 minutes.
The procedure for the study is as follows: (a) 3 minutes for introduction, (b) 10 minutes for the
self-regulation questionnaire and demographic questionnaire, (c) 10 minutes for instruction of
knowledge mapping, (d) 10 minutes for game introduction, (e) 15 minutes for first game playing,
(f) 7 minutes for the first knowledge map drawing (intermediate), (g) 4 minutes for the first
problem-solving strategy questions (intermediate), (h) 1 minute for task completion questionnaire,
(i) 15 minutes for the second game-playing (j) 7 minutes for the second knowledge map drawing
(post), (k) 4 minutes for the second problem-solving strategy questions (post), (l) 1 minute for task
completion questionnaire, (5) 3 minutes for follow up, (n) up to 30 minutes of free optional play
time.
13. Drugs and Devices
None
14. Data Storage and Confidentiality
The data will be kept in a locked filing cabinet in our research office. Only Dr. O’Neil and Mr.
Richard Wainess will have access to the raw data. Assigned ID numbers for participants, instead
of their names, will be used. The research office is in a locked office at USC. The raw data with
names will be destroyed at the end of the study.
RISK/BENEFIT ASSESSMENT
15. Potential Risks:
There is a remote chance a participant might feel dizziness from playing a 3D game. This occurs
rarely, and is more likely with females than males. The effect is minor and temporary. There are
no lasting effects. Participants may stop participating at any time.
16. Risk Classification:
Minimal.
17. Protection Against Risks:
N/A
18. Data Safety Monitoring Plan:
Only Dr. O’Neil and Mr. Richard Wainess will have access to the data collected. Assigned ID
numbers for participants, instead of their names, will be used.
19. Potential Benefits to the Subject and/or Society:
There are no direct benefits to the subject for participating in this study. There may be important
benefits to educational instruction as a result of this study. With video games becoming so
embedded in cultures and societies worldwide, and with the intrinsic appeal of games, especially
to young learners, it is only natural to look to video games as a platform for delivering instruction.
Yet, to be an effective platform, it is imperative that researchers discover the methods that can
best aid students while learning in 3D environments. This study examines one of those methods.
20. Therapeutic Alternatives:
N/A
21. Risk/Benefit Relationship:
IRB Application Section II (Rev. 4/2003) – Protocol Detail
Page 3 of 6
N/A
FINANCIAL OBLIGATIONS AND COMPENSATION
22. Financial Obligations of the Subject:
None
23. Financial Compensation for Participation:
$15, regardless of whether the participant completes the study.
SUBJECT IDENTIFICATION, RECRUITMENT AND CONSENT/ASSENT
24. Method of Subject Identification and Recruitment:
Subjects will be recruited through posting of the study flyer at various, approved, campus
locations, and by emailing the flyer through several USC listProcs, including imc-l@usc.edu,
edtech-l@usc.edu, and scfx@usc.edu with approval from the listProc owners.
25. Competing Protocols:
N/A
26. Subject Competency:
All subjects will be competent to give informed consent.
27. Process of Informed Consent:
Subjects will receive an electronic version of the consent form upon request to participate in the
study and will be asked to review the document prior to arrival at the experiment. During the 3
minute introductory phase of the experiment, Richard Wainess will review the purpose and process
of the study as described in the consent form. Richard Wainess will also answer any questions
participants may have regarding the consent form and the study, and will require each participant to
print his or her name, and sign and date the consent form, in order to participate in the study. On a
“participant number form” containing two columns, one for the participant name and one
containing a unique, predefined randomly generated two-digit number, Richard Wainess will
record the participant’s name and write the associated number on each form handed to the
participant. The participant number form will remain confidential and only accessible to the
researchers.
28. Subject or Representative Comprehension:
Prior to signing the consent form, and after answering all questions subjects have regarding the
form and the study, subjects will be told that signing the form indicates their understanding of the
nature of the study and the study procedures. Subjects will also be reminded that participation is
voluntary and they may leave without signing and without repercussions for not signing, except
that the $15 is only paid to participants are they are not considered participants until they sign the
consent form.
29. Information Purposely Withheld (Deception):
None
30. Consent/Assent Forms:
Adult consent form
31. Documentation of Consent/Assent:
Dr. Harold O’Neil and Mr. Richard Wainess will keep all the signed consent forms.
32. Waiver of Informed Consent:
N/A
IRB Application Section II (Rev. 4/2003) – Protocol Detail
Page 4 of 6
33. Waiver of the HIPAA Authorization:
a. Recruitment/Screening Only
b. Research Subject
N/A
34. Exception from Informed Consent Requirements for Emergency Research:
N/A
35. Waiver of Written Informed Consent:
N/A
36. Proprietary Interest Disclosure:
N/A
37. Privileges/Certifications and Licenses:
N/A
38. Industry Studies:
N/A
39. Bibliography:
Brunken, R., Plass, J. L., & Leutner, D. (2003). Direct measurement of cognitive load in
multimedia learning. Educational Psychologist 38(1), 53-61.
Chalmers, P. A. (2003). The role of cognitive theory in human-computer interface. Computers in
Human Behavior, 19, 593-607.
Cutmore, T. R. H., Hine, T. J., Maberly, K. J., Langford, N. M., & Hawgood, G. (2000).
Cognitive and gender factors influencing navigation in a virtual environment. International
Journal of Human-Computer Studies, 53, 223-249.
de Jong, T., & van Joolingen, W. R. (1998). Scientific discovery learning with computer
simulations of conceptual domains. Review of Educational Research, 68, 179-202.
Garris, R., Ahlers, R., & Driskell, J. E. (2002). Games, motivation, and learning: A research and
practice model. Simulation & Gaming, 33(4), 441-467.
Harp, S. F., & Mayer, R. E. (1998). How seductive details do their damage: A theory of cognitive
interest in science learning. Journal of Educational Psychology, 90(3), 414-434.
Lee, J. (1999). Effectiveness of computer-based instructional simulation: A meta analysis.
International Journal of Instructional Media, 26(1), 71-85.
Mayer, R. E., Mautone, P., & Prothero, W. (2002). Pictorial aids for learning by doing in a
multimedia geology simulation game. Journal of Educational Psychology, 94(1), 171-185.
Paas, F., Tuovinen, J. E., Tabbers, H., & Van Gerven, P. W. M. (2003). Cognitive load
measurement as a means to advance cognitive load theory. Educational Psychologist, 38(1), 6371.
IRB Application Section II (Rev. 4/2003) – Protocol Detail
Page 5 of 6
Renkl, A., & Atkinson, R. K. (2003). Structuring the transition from example study to problem
solving in cognitive skill acquisition: A cognitive load perspective. Educational Psychologist,
38(1), 13-22.
Thiagarajan, S. (1998, Sept/October). The myths and realities of simulations in performance
technology. Educational Technology, 38(4), 35-41.
Wolfe, J. (1997, December). The effectiveness of business games in strategic management course
work [Electronic Version]. Simulation & Gaming Special Issue: Teaching Strategic Management,
28(4), 360-376.
IRB Application Section II (Rev. 4/2003) – Protocol Detail
Page 6 of 6