UNIVERSITY PARK INSTITUTIONAL REVIEW BOARD APPLICATION
SECTION II
TEMPLATE REV. 4/2003
Original Date:
Revision Date:
TITLE OF PROJECT:
The Effect of Navigation Maps on Problem Solving Tasks Instantiated in a Computer-Based Video
Game
PURPOSE OF THE STUDY AND BACKGROUND
1. Specific Aims and Purpose of the Study:
Our programmatic research is directed towards answering the following questions. (1) Will the
problem solving performance within a video game (i.e., SafeCracker) for participants who use a
navigation map be greater than for those participants who do not use the navigation map? (2)
Will continued motivation be greater for participants who use the navigation map than for those
who do not use the navigation map?
The study will primarily focus on the impact of the navigation map on game on problem
solving. The problem solving assessment is composed of three components: content
understanding, problem-solving strategies, and self-regulation. Content understanding or domain
knowledge is measured by a concept map. Problem-solving strategies are measured by domain
specific problem-solving questions. Self-regulation is measured by a trait thinking
questionnaire, assessing planning, self-checking, effort, and self-efficacy.
Participants will be tested individually via computer systems for the computer assessed version.
The entire session will take one 90 minute period, plus optional free play time of up to 30 minutes:
(a) 3 minutes for introduction, (b) 8 minutes for the self-regulation questionnaire and demographic
questionnaire, (c) 10 minutes for instruction of knowledge mapping, (d) 10 minutes for game
introduction, (e) 5 minutes for the first knowledge map drawing (pre), (f) 3 minutes for problemsolving strategy questions, (g) 15 minutes for first game playing, (h) 5 minutes for the second
knowledge map drawing (intermediate), (i) 3 minutes for the first problem-solving strategy
questions (intermediate), (j) 30 seconds for task completion questionnaire, (k) 15 minutes for the
second game-playing (l) 5 minutes for the second knowledge map drawing (post), (m) 3 minutes
for the second problem-solving strategy questions (post), (n) 30 seconds for task completion
questionnaire, (o) 4 minutes for debriefing, (p) up to 30 minutes of free optional play time.
Questionnaires will be administered to evaluate participants’ self regulation. When drawing the
concept map, participants express their content understanding obtained from the game. The domain
specific problem solving questions will assess participants’ problem solving strategies gained from
playing the game.
2. Background and Significance:
Research into the effectiveness of games and simulations as educational media has been met with
mixed reviews (de Jong & van Joolingen, 1998; Garris, Ahlers, & Driskell, 2002). It has been
suggested that the lack of consensus can be attributed to weaknesses in instructional strategies
embedded in the media and issues related to cognitive load (Chalmers, 2003; Cutmore, Hine,
Maberly, Langford, & Hawgood, 2000; Lee, 1999; Thiagarajan, 1998; Wolfe, 1997). A major
University of Southern California University Park IRB
Page 1 of 5
instructional issue in learning by doing within simulated environments concerns the proper type
of guidance, that is, how best to create cognitive apprenticeship (Mayer, Mautone, & Prothero,
2002).
A virtual environment creates a number of issues with regards to learning. Problem-solving within
a virtual environment involves not only the cognitive load associated with the to-be-learned
material (referred to as intrinsic cognitive load: Paas, Tuovinen, Tabbers, Van Gerven, 2003), it
also includes cognitive load related to the visual nature of the environment (referred to as
extraneous cognitive load: Brunken, Plass, & Leutner; Harp & Mayer, 1998), as well as
navigating within the environment—either germane cognitive load or extraneous cognitive load,
depending on the relationship of the navigation to the learning task (Renkl, & Atkinson, 2003).
An important goal of instructional design within these immersive environments involves
determining methods for reducing the extraneous cognitive load and/or germane cognitive load,
thereby providing more working memory capacity for intrinsic cognitive load (Brunken et al.,
2003). This study will examine the reduction of cognitive load, by providing graphical scaffolding
in the form of a navigation map, to determine if this can result in better performance outcomes as
reflected in retention and transfer (Paas et al., 2003) in a game environment.
3. Progress Report/Preliminary Studies:
N/A
CHARACTERISTICS OF THE STUDY SUBJECT POPULATION
4. Target Accrual:
60-65
5. Age Range of Adult Subjects:
18 or older
6. Age Range of Minor Subjects:
N/A
7. Gender of the Subjects:
Male/Female
8. Racial and Ethnic Origin:
Non-specific
9. Inclusion of Special Classes or Vulnerable Subjects:
None
10. Inclusion Criteria:
The subject must be a college student, a graduate student, or a graduate.
11. Exclusion Criteria:
Anyone who is not a college student, a graduate student, or a graduate. Anyone who has played the
video game SafeCracker, prior to participation in the study.
METHODS AND PROCEDURES
12. Research Design and Methods Applied to Human Subjects:
Participants will be tested individually via computer systems for the computer assessed version.
The entire session will take one 90 minute period, plus optional free play time of up to 30 minutes:
IRB Application Section II (Rev. 4/2003) – Protocol Detail
Page 2 of 5
(a) 3 minutes for introduction, (b) 8 minutes for the self-regulation questionnaire and demographic
questionnaire, (c) 10 minutes for instruction of knowledge mapping, (d) 10 minutes for game
introduction, (e) 5 minutes for the first knowledge map drawing (pre), (f) 3 minutes for problemsolving strategy questions, (g) 15 minutes for first game playing, (h) 5 minutes for the second
knowledge map drawing (intermediate), (i) 3 minutes for the first problem-solving strategy
questions (intermediate), (j) 30 seconds for task completion questionnaire, (k) 15 minutes for the
second game-playing (l) 5 minutes for the second knowledge map drawing (post), (m) 3 minutes
for the second problem-solving strategy questions (post), (n) 30 seconds for task completion
questionnaire, (o) 4 minutes for debriefing, (p) up to 30 minutes of free optional play time.
Questionnaires will be administered to evaluate participants’ self regulation. When drawing the
concept map, participants express their content understanding obtained from the game. The domain
specific problem solving questions will assess participants’ problem solving strategies gained from
playing the game.
13. Drugs and Devices
None
14. Data Storage and Confidentiality
The data will be kept in a locked filing cabinet in our research office. Only Dr. O’Neil and Mr.
Richard Wainess will have access to the raw data. The research office is in a locked office at USC.
The raw data with names will be destroyed at the end of the study.
RISK/BENEFIT ASSESSMENT
15. Potential Risks:
Minimal
16. Risk Classification:
N/A
17. Protection Against Risks:
N/A
18. Data Safety Monitoring Plan:
Only Dr. O’Neil and Mr. Richard Wainess will have access to the data collected. Assigned ID
numbers for participants instead of their names will be used.
19. Potential Benefits to the Subject and/or Society:
The potential benefits to science or society expected from this research involve adding to our
knowledge of how to assess the training effectiveness of a computer game and problem solving in
computer based environments. Training with computer games is the trend, so a scientifically
supported framework of evaluation needs to be built. In addition, problem solving is a critical skill
in various job settings, but for the time being, there are no methods to evaluate of such skills. The
participants will gain some experience in taking these kinds of assessments. Furthermore, such
experience may be useful for future jobs.
20. Therapeutic Alternatives:
N/A
21. Risk/Benefit Relationship:
N/A
FINANCIAL OBLIGATIONS AND COMPENSATION
IRB Application Section II (Rev. 4/2003) – Protocol Detail
Page 3 of 5
22. Financial Obligations of the Subject:
None
23. Financial Compensation for Participation:
None
SUBJECT IDENTIFICATION, RECRUITMENT AND CONSENT/ASSENT
24. Method of Subject Identification and Recruitment:
Only Dr. O’Neil and Mr. Richard Wainess will have access to the data collected. Although names
will be used, ID numbers will be assigned and the coding sheet of names/ID numbers will be
destroyed.
25. Competing Protocols:
N/A
26. Subject Competency:
Yes
27. Process of Informed Consent:
An informed consent will be given to the subject
28. Subject or Representative Comprehension:
The subjects have the ability to comprehend everything in the study and in the consent form.
29. Information Purposely Withheld (Deception):
No
30. Consent/Assent Forms:
adult consent form
31. Documentation of Consent/Assent:
Dr. Harold O’Neil and Mr. Richard Wainess keep all the signed consent form.
32. Waiver of Informed Consent:
N/A
33. Waiver of the HIPAA Authorization:
a. Recruitment/Screening Only
b. Research Subject
N/A
34. Exception from Informed Consent Requirements for Emergency Research:
N/A
35. Waiver of Written Informed Consent:
N/A
36. Proprietary Interest Disclosure:
ONR
37. Privileges/Certifications and Licenses:
N/A
38. Industry Studies:
N/A
39. Bibliography:
Brunken, R., Plass, J. L., & Leutner, D. (2003). Direct measurement of cognitive load in
multimedia learning. Educational Psychologist 38(1), 53-61.
IRB Application Section II (Rev. 4/2003) – Protocol Detail
Page 4 of 5
Chalmers, P. A. (2003). The role of cognitive theory in human-computer interface. Computers in
Human Behavior, 19, 593-607.
Cutmore, T. R. H., Hine, T. J., Maberly, K. J., Langford, N. M., & Hawgood, G. (2000).
Cognitive and gender factors influencing navigation in a virtual environment. International
Journal of Human-Computer Studies, 53, 223-249.
de Jong, T., & van Joolingen, W. R. (1998). Scientific discovery learning with computer
simulations of conceptual domains. Review of Educational Research, 68, 179-202.
Garris, R., Ahlers, R., & Driskell, J. E. (2002). Games, motivation, and learning: A research and
practice model. Simulation & Gaming, 33(4), 441-467.
Harp, S. F., & Mayer, R. E. (1998). How seductive details do their damage: A theory of cognitive
interest in science learning. Journal of Educational Psychology, 90(3), 414-434.
Lee, J. (1999). Effectiveness of computer-based instructional simulation: A meta analysis.
International Journal of Instructional Media, 26(1), 71-85.
Mayer, R. E., Mautone, P., & Prothero, W. (2002). Pictorial aids for learning by doing in a
multimedia geology simulation game. Journal of Educational Psychology, 94(1), 171-185.
Paas, F., Tuovinen, J. E., Tabbers, H., & Van Gerven, P. W. M. (2003). Cognitive load
measurement as a means to advance cognitive load theory. Educational Psychologist, 38(1), 6371.
Renkl, A., & Atkinson, R. K. (2003). Structuring the transition from example study to problem
solving in cognitive skill acquisition: A cognitive load perspective. Educational Psychologist,
38(1), 13-22.
Thiagarajan, S. (1998, Sept/October). The myths and realities of simulations in performance
technology. Educational Technology, 38(4), 35-41.
Wolfe, J. (1997, December). The effectiveness of business games in strategic management course
work [Electronic Version]. Simulation & Gaming Special Issue: Teaching Strategic Management,
28(4), 360-376.
IRB Application Section II (Rev. 4/2003) – Protocol Detail
Page 5 of 5