Version 0.5
25. November 2003
Review of the research on educational
usage of games
Simon Egenfeldt-Nielsen
IT-University Copenhagen
Glentevej 67
DK-2400 NV Copenhagen
+45 40 10 79 69
egenfeldt@game-research.com
Abstract
The goal of this paper is to make it possible to overview the area of educational games
before embarking on new research. The paper examines the current research evidence
for and against the usage of games for educational purposes. The paper will also
outline two important areas that have so far been identified as important for the
effectiveness of games: Learning environment and personal learning factors. It is
concluded that these areas are very important areas but often they are not well
documented nor researched to a large extent.
In relation to the effect of using games it is concluded that there is currently no evidence
for a better or worse learning outcome, when games are used. Although there is
indication that retention might be better when using games for learning.
Keywords
Games, computer games, education, learning, review, simulation, effectiveness
Version 0.5
25. November 2003
Introduction
There should be no doubt in our mind that we can learn something from games just like
other activities we engage in. The discussion is rather what we learn and how effective
the learning process is. Following this discussion another question often arises. Namely
whether games have a place in educational settings, and if so under what conditions in
relation to for example subject, setting, and age group. Although this question is not
new it remains central to the study of simulation and games for learning (see for
example Boocock & Schild, 1968).
Although this paper focuses on non-electronic games it is my hope that this article can
be used to transfer some of the research from educational games to the younger
research field educational computer games. There is good reason to believe that we
face similar problems in the two fields (Egenfeldt-Nielsen, 2004).
In the following I will not distinguish between simulations and games on an overall plan,
as I believe the majority of simulations that have been used over the years can to some
degree be described as games. Hereby I follow the distinction by Egenfeldt-Nielsen &
Smith (2000) where simulation and games is separated by the existence of goals.
Simulation games do not have explicit goals as such, whereas these are central to
games. Furthermore it seems that setting up goals can transfer any simulation into a
game. This is a general definition which is useful for this article although it somewhat
simplifies the distinction between simulations and games.
When we combine the research areas learning, education, games, and simulation we
have a very cross-disciplinary discipline, which inherit several tough and controversial
research questions from their respective areas. One of the main problems of presenting
the results below is that the evaluation methods in relation to games is a problem in
themselves, where it has been questioned if we can use traditional methods for
measuring learning outcome (Druckman, 1995). I will try to address some of these
concerns towards the end of the paper.
Some researchers claim that we simply learn to play the game in question while other
subtler argues for a transfer of knowledge, where the elements we experience within in
the game context can be used in other areas of our life. This question is interesting and
relevant but is not within the scope of this paper. In this paper I assume that we are
actually in some way capable of transferring knowledge between different contexts.
In summary this paper will survey the research that has so far been carried out into the
effectiveness of games. The focus is on providing an overview rather than going in
detail with specific studies.
Version 0.5
25. November 2003
A frame for reviewing learning through games
The following examination of different important aspects of games promoting learning is
based on the categories in earlier review studies. It has not been possible to go into
detail with separate studies but instead I aim to present an overview of the most
important review articles over the years (Wentworth & Lewis, 1973; Bredemeier &
Greenblat, 1981; Van Sickle, 1986; Dorn, 1989; Clegg, 1991; Randel et al., 1992;
Dempsey et.al, 1996). It should be noted that the reviews are quite dated, which is a
sign of the decline of research in non-electronic educational games in favour of
research on educational potential of computer games (Egenfeldt-Nielsen, 2003). In
some places I will use articles of newer date to present areas not covered sufficiently in
existing reviews. The review has been split into three areas. Each of these areas will be
presented and discussed in the paper to give a status of the research field.
1. Learning environment: What properties of a learning environment have a
bearing on the learning outcome, and what variables are important.
2. Personal learning factors: What personal factors can play a role for the
learning outcome of a specific game experience, and how significant is the
impact.
3. Learning outcome: What are the effects of using games for learning within
different areas and in relation to different domains of learning.
Before entering into the specifics of these areas it is worth considering the more overall
factors like the type of games used, and within what subjects. It seems obvious that
games could lend themselves better to some subjects, and that not all game types have
been equally used. Early on there were few been studies comparing the advantages
and differences between different game genres (Roberts, 1976) but from the literature it
is clear that simulation type is preferred. Research that compares the suitability of
different types of games for learning is still limited. In a search on what game types
were encountered in articles on simulation and gaming Dempsey et al. (1996) found 43
simulation games, 26 other, 10 adventure, 4 puzzles, and 1 experimental. They
attribute the large number of simulations to the history of the research, which grows out
of business1 and military training (Roberts, 1973). The problem with their distinction is
that the simulation category is quite broad, as simulation is in a majority of games an
almost intrinsic property.
1
Especially business education have taken games to their heart perhaps best
exemplified with the following web-site that list 140 games specifically developed for
business http://www.marietta.edu/~delemeeg/games/
Version 0.5
25. November 2003
The learning environment
The role of the learning environment has been studied from different angles focusing on
the significance of subject area, education level, motivation, the use of debriefing,
administration, and instructor type.
It seems that games have from the start especially been used within the areas social
sciences, math, business, and military (Dukes & Seidner, 1978; Butler, Markulis &
Strang, 1988; Prensky, 2001), which are also the areas where the research efforts has
been greatest. Randel et al. (1992) finds that social studies top the list with 46
conducted studies, 8 studies within math, 6 in language arts, 3 in biology, and 1 in
physics.
Over the years the use of games has spread to all areas of the education system but
with primary and secondary education lacking behind compared to tertiary education
(Ellington, 1995). This corresponds with Dempsey et al. (1996) that find that the use of
games in adult learning is the most researched area. The success of games in tertiary
education might be explained as a consequence of, what is socially acceptable.
Research into adults using games for learning is probably perceived as more serious as
adults don’t play for fun. This is serious research of a serious tool for learning.
Furthermore the teaching of adults with games might be conceived as more serious as
adults according to Sutton-Smith & Kelly-Byrne (1984) are not expected to play.
Therefore games are constructed as a serious activity for adults when in an educational
setting. For children on the other hand teaching with games is perceived as nothing
else than play - and do they really learn anything from that? The difference cannot be
explained due to better effect in adult learning as Dempsey et.al. (1996) do not find any
indication that games are more effective for adult teaching.
The importance of motivation and interest
It has be claimed that the overall atmosphere of the learning environment improve when
games are used, a claim which Seidner (1978), Bredemeier & Greenblat (1981), and
Dorn (1989) support, however the evidence is not convincing. Still the idea of an
improvement in the overall atmosphere ties well into one of the most significant
variables concerning the learning environment of games: The increase in students’
motivation and interest. This factor is identified as far back as 1968 and continues to be
central. The majority of researchers like Boocock, & Schild (1968), Wentworth & Lewis
(1973), Coleman et al. (1973), Seidner (1978), and Bredemeier & Greenblat (1981).
Dorn (1989), Clegg (1991) and Randel et al. (1992), support this claim. However
Version 0.5
25. November 2003
despite the common agreement of games increasing motivation and interest, it still
remains to be seen, what the specific implications are of this increase and whether the
increased motivation relates to a better learning experience. Dorn (1989) questions
whether the increased motivation for specifically games holds true when you compare
games with more active teaching methods like case studies.
Debriefing – an indispensable tool
Debriefing is another classic issue, which from the start has been perceived as an
important element in the learning environment although the early Hopkins Games
Program (Coleman et al, 1973) did not find that a class discussion after the game
increased the learning outcome, however this may be due to a weak guided debriefing
process or limited capability of the teacher that conducted the debriefing (Seidner,
1978). Later research support debriefing for example Bredemeier & Greenblat
(1981:310), which state that debriefing is “essential for maximum (or even correct)
learning to occur”. Linda Costigan Lederman (1984:426-429) follows up on the
importance of debriefing for correct learning stressing the need for the teacher to
validate the learning experience, and check how reliable the outcome is. Furthermore
she emphasize the need to assess the exercise from a cost-benefit perspective, an
element that is still highly relevant, as we need to compare the learning outcome with
the extra time spend on for example learning the game interface, setting up the game,
and additional time for teachers preparation. The debriefing is a way to increase
reflection over games that can sometimes be quite hectic (Thatcher, 1990).
According to Bredemeier & Greenblat (1981) and Lederman & Fumitoshi (1995) the
importance of debriefing is well recognized but the research is limited. In a recent article
Charles Petranek (2000) backs up this claim and points to the lack of written debriefing.
He argues that written debriefing is the next natural step in a learning process, and it is
surprising that research into written debriefing is almost non-existing.
The scourge of the setting
The setting of games is usually thought of as the school, and this holds certain
problems related to the quite rigid structure of the school system. Francois Saegesser
(1981; 1984) and Coleman et al. (1973) points out that the time schedule, physical
space, learning theory, and existing role scripts hinders the use of games, and this
potentially influence the learning outcome. The influence of these factors are hard to
access, however the different roles administration and instructors play, should alert us
to the implications of the overall structure in the school system. Bredemeier & Greenblat
Version 0.5
25. November 2003
(1981) and Dorn (1989) state that the attitude of the instructor toward games influences
the outcome, and that the instructor’s knowledge and skill in using the game is also a
factor. Furthermore the concrete use of the game can change the outcome if the
instructor uses a variation of the gameplay or introduces the game differently.
The importance of the learning setting has been somewhat neglected in the research
over the years perhaps as a consequences of the more rich hunting fields in adult
learning especially business and military training. Here the constraints and traditions of
learning are less tainted by time.
The instructors role
Some demands can be made on the instructor for achieving the best learning outcome.
Dorn (1989) outline some of them, and in accordance with this review I have added
some. The question of how to use games have not been discussed in this paper but it
worth noting that according to Dorn (1989) games should not be the only teaching
method in a course, and the game sessions should be supported by other teaching
methods and have a clear aim. Depending on the use and timing of games in a course,
they can serve different purposes for example games in the start of a course can serve
to introduce theory and concept while giving the students a way to know each other.
More specifically the instructor should have some experience with the games used, be
skilled in administering the game, have a desire to use the game, keep in mind the role
scripts in the class, be able to perform debriefing, evaluate different games, and be
aware of the physical limits with regards to time and space in school. These seem like
basic and reasonable demands, however they may prove much harder to implement in
real life. These recommendations are also in line with Charles Elders (1973) findings.
The complexity that opens up with the many variables in the learning environment are
staggering and quite overwhelming for anyone with a practical focus although some
seem convinced these can be overcome (Dorn, 1989).
Personal learning factors
The role of personal characteristics has been examined to some degree although
Dempsey et al. (1996:12) finds that in reality studies are “very unclear in reporting these
characteristics”. In the review the most frequently reported variables were gender, age,
academic ability, and to some degree race. This is quite interesting as the early work in
the field stress the importance of individual differences (Greenblat, 1981), and the
importance of examining the sample to avoid method problems related to matched
Version 0.5
25. November 2003
samples (Remus, 1981), which is often a given experimental design as you must
conduct the study within the existing class structure in schools. The fact that basic
factors like age, race, gender, and academic ability is only cited in approximately 25%
of the studies indicate the problems are not yet resolved.
One area that has consistently received attention is whether games favour the students
with the best academic abilities or is perhaps a mean to reach the students with weaker
academic abilities. According to Coleman et al. (1973) the games are more effective
with students with high academic ability, however Bredemeier & Greenblat (1981) finds
more mixed results and points out that games may be able to reach less advantaged
student groups but this depends on other factors as well. They especially point out that
the prior attitude of the student towards the game plays a role.
Other indications of the role of personal factors are the structure of the group in relation
to size, organization, and relations between students (Clegg, 1991). The argument is
that the composition of the group facilitates different learning experiences and dynamics
in both the environment and for the specific student for example winners and losers of
the game have very different experiences (Bredemeier & Greenblat, 1981). It seems
that the initial group cohesion is quite important for the performance in a game
environment, and remains relatively stable as the game progress (Wellington & Faria,
1996). In the study by Wellington & Faria (1996) the initial attitude of the students is
found to be of limited consequence for the performance, whereas group cohesion is
central.
One area that has received little attention is the importance of earlier experience with
games, which could be expected to play a role for learning outcome. Knowledge and
experience with playing games would make it possible for the student to concentrate on
playing the game, and spending less time mastering a new learning form.
Learning outcome
This is one of the areas, which have received the most attention over the years; how
can we measure the learning effect of games on users. As I have implied earlier this
question is way to broad, and needs to be qualified with: “For what purpose, under what
conditions, and how can we be sure.” (Bredemeier & Greenblat, 1981:307). This is what
I have tried to clear the way for in the sections above, however I will still try to give a
general verdict on the learning outcome associated with games in the following.
The claims for what games can do is very long even in 1981 when Greenblat (1981)
sums it up. Greenblat’s list is too long to reproduce here but in short forms it spans two
pages with 30 different claims.
Version 0.5
25. November 2003
A Procrustean bed: Assessing learning outcome
The 30 claims and other claims over the years 2 have been hard to document due to
problems with measuring learning in a school context and to construct methods for
measuring in accordance with existing dominating learning theories. Also it has been
pointed out that several studies in general have method flaws (Dorn, 1989). These
problems have been clear from early on but continue to influence the research.
Bredemeier & Greenblat (1981) and Dorn (1989) points out that we should be aware of
what basic learning theory we ascribe to, and what assessment is feasible to confirm
learning outcome within our theoretical frame. Saegesser (1981) points out that existing
learning theories in schools put external signs of learning over more subtle forms.
Although these notions of warning are more than 20 years old, they still hold true
although the theoretical landscape of learning have changed significantly (Gardner,
1983). We still run the risk of not realising the full potential of games if we try to put
them into a Procrustean bed. On the other hand we need to be specific about the
learning outcome we expect, and find other ways that we can measure learning.
Otherwise we run the risk of the learning being repetitive, undocumented, confusing
and pointing in different directions, which is a very real risk, when using games for
teaching (Elder, 1973).
The assessment of learning has given rise to numerous problems over the years. One
of the most persistent is the lack of taking into consideration the most basic factors
when conducting a study. According to Joseph Wolfe and David Crookall (1998) these
problems are partly related to the research area being quite eclectic. This has led too
less rigorous standards for assessment.
In line with Elder (1973), Bredemeier & Greenblat (1981) state that three areas need to
be measured to gauge learning outcome in a given game. The overall variable is
substantive learning, which can be both learning about a subject or about oneself
(cognitive and affective). When assessment of learning outcome is divided into
cognitive and affective we can distinguish between different domains of knowledge and
learning methods. The split between cognitive and affective is also sometimes seen as
the division of specific skills and more general principles, concepts, and orientations.
The problems with distinguishing between these two areas have influenced game
research from the start and the problem is shared with other educational research
(Elder, 1973).
The two other areas that as a minimum should be assessed are those having a bearing
on the overall variable learning outcome. These are motivation to learn and the relation
2
For example the loose claims at the Educational Simulations web-site
http://www.creativeteachingsite.com/edusims.html
Version 0.5
25. November 2003
between students and teacher. Most of these variables are taken into consideration in
serious new studies for example the study of Pomp and Circumstances by Meguni
Kashibuchi and Akira Sakamoto (2001).
The outcome of cognitive and affective learning
The reviews are not consistent in their conclusions on learning outcome but in general it
can be concluded that: In some instances games seem to be more appropriate for
affective learning, while cognitive learning in general is found to be as good as other
teaching methods (excluding retention over time) (Bredemeier & Greenblat, 1981; Dorn,
1989; Van Sickle, 1986, Randel et.al, 1992).
The review by Dempsey et al. (1996) following Gagne’s taxonomy3 shows that problem
solving is the most researched learning outcome followed by attitudes. This imply that in
general much of the research have been aimed at identifying learning outcome in more
overall competences, and not necessarily the learning outcome of facts and concepts.
This also ties in well with the overweight of research in social sciences, where problem
solving and attitudes have traditionally been a focus (Abt, 1968:70-71).
The success of math games reported by Randel et al. (1992) to some degree runs
counter to this, as math is a subject that learns very specific skills. Randel et.al. (1992)
review concludes that the most successful learning outcome is achieved in games with
very specific required skills. This implies that it may be relevant to revisit games for
learning more specific skills for example in algebra, physics, chemistry or grammar
rules.
The increased retention over time of learning appears to be one of the most consistent
finding within the research into the potential of games for learning (Bredemeier &
Greenblat, 1981; Randel et al., 1992; Van Sickle, 1986), however Dorn (1989)
challenges this claim citing older studies that show no difference in retention over time.
However the newer and more thorough study by Randel et al. (1992) supports the claim
for better retention over time. Although not all the studies in the meta-analysis by
Randel et al. (1992) examine retention over time the ones that did, find a better
retention over time compared to traditional teaching methods.
3
Attitude, motor skills, cognitive strategy, problem solving, rules, defined concepts,
verbal information, other.
Version 0.5
25. November 2003
Conclusion
We have seen that games are used in very different subjects with varying degree of
success. Some of the popular areas for games have been business, math, social
studies, science, history, geography, and military. You may end up concluding that
games have been used in almost any imaginable setting, subject, and age group, which
is not entirely wrong. Within all these areas games are used for different purposes and
it is still unclear what exactly defines games.
We have indications that this may be a worthwhile area to study but we still need to
become much more specific in terms of what games we are talking about. Although
research into the specifics of each game and game types is limited, I firmly believe that
a lot of the contradictory results stem from too little attention to the game in question. It
does matter what kind of game we are studying, and more importantly we can have a
large number of badly designed games. We should be careful not to equate a specific
bad or extraordinary game with a general claim for using games for learning. A lot of the
early studies on learning and games used games that were developed by fiery souls
and with little training in designing games. It was learning by doing and some of the
results from these first educational games are still with us today. These results may not
show problems with games per se but rather with badly designed games.
There are still a lot of problems left, when we look at learning and games but it would be
a shame if we didn’t build on the rich results from the early days. The area stands a
good chance of revitalization if it lends itself more to computer game research that is
growing explosively these years.
References

Abt, Clark (1968). Games for Learning. In: Boocock, Sarane S. & Schild, E.O.
(1968)(eds.). Simulation Games in Learning. London: Sage Publications.

Boocock, Sarane S. & Schild, E.O. (1968)(eds.). Simulation Games in Learning.
London: Sage Publications.

Bredemeier, M.E., & Greenblat, C.S. (1981). The educational effectiveness of
simulation games: A synthesis of findings. Simulation & Games, Vol. 12, No. 3:
307-331.

Butler, Markulis & Strang (1988). Where are we? An Analysis of the Methods and
Focus of The Research on Simulation Gaming. Simulation & Games, Vol. 19, No.
1: 3-26.

Cathy Greenblat & Richard E Duke (1981)(eds.). Gaming-Simulation: Rationale
Applications. London: Sage Publications.
Version 0.5
25. November 2003

Clegg, A.A. (1991). Games and simulations in social studies education. In:
Shaver, J. P. (1991)(eds.). Handbook of research on social studies teaching and
learning. New York: Macmillan.

Coleman, J. S., et al. 1973. The Hopkins' Games Program: Conclusions from
Seven Years of Research. Educational Researcher 2:3-7.

Dempsey, John V., Rasmussen, Karen & Lucassen, Barbara (1996). The
Instructional Gaming Literature: Implications and 99 Sources. University of South
Alabama. Technical Report 96-1.

Dorn, D. S. 1989. Simulation Games: One More Tool On the Pedagogical Shelf.
Teaching Sociology Vol. 17:1-18.

Druckman, Daniel (1995). The Educational Effectiveness. In: Crookall, David &
Arai, Kiyoshi (1995)(eds.). Simulation and gaming across disciplines and cultures.
London: Sage Publications.

Duke, Richard E. & Seidner, Constance J. (1975)(eds.). Learning with simulations
and games. London: Sage Publications.

Egenfeldt-Nielsen, Simon & Smith, Jonas H. (2000). Den digitale leg – om børn
og computerspil [Digital Play – About children and computer games].
Copenhagen: Hans Reitzels Forlag.

Egenfeldt-Nielsen, Simon (2003). The current status of the research communities
in games and computer games. Work in Progress.

Egenfeldt-Nielsen, Simon (2004). Practical barriers in using educational computer
games. On the Horizon. Under preparation.

Elder, Charles (1973). Problems in the Structure and Use of Educational
Simulation. Sociology of Education, Vol. 46: 335-354.

Ellington, Henry (1995). The Future of Simulation/Gaming in Britain. In: Crookall,
David & Arai, Kiyoshi (1995)(eds.). Simulation and gaming across disciplines and
cultures. London: Sage Publications.

Gardner, Howard. (1983) Frames of mind: the theory of multiple intelligences.
New York: Basic Books.

Greenblat (1981). Teaching with Simulation Games: A review of Claims and
Evidence. In: Duke, Richard E. & Greenblat, Cathy (1981)(eds.). Principles of
Practice of Gaming-Simulation. London: Sage Publications.

Kashibuchi, Meguni & Sakamoto, Akira (2001). The educational effectiveness of
a simulation/game in sex education. Simulation & Gaming, Vol. 32, No. 3: 331343.

Lederman, Linda C. & Fumitoshi, Kato (1995). Debriefing the Debriefing Process:
A new look. In: Crookall, David & Arai, Kiyoshi (1995)(eds.). Simulation and
gaming across disciplines and cultures. London: Sage Publications.

Lederman, Linda C. (1984). Debriefing: A Critical Reexamination of the Post
experience Analytic Process with Implications for Its Effective Use. Simulation &
Games, Vol. 15, No. 4:415-431.

Leemkuil, Henny, Jong, Ton de & Ootes, Susanne (2000). Review of educational
use of games and simulations. Twente: University of Twente.
Version 0.5
25. November 2003

Loftus, Geoffrey & Loftus, Elizabeth (1983). Mind at Play: The Psychology of
Video Games. New York; Basic Books.

Prensky, Marc (2001). Digital Game-Based Learning. New York: McGraw-Hill

Randel, J. M., B. A. Morris, C. D. Wetzel, and B. V. Whitehill (1992). The
Effectiveness of Games for Educational Purposes: A Review of Recent
Research. Simulation & Gaming Vol. 23, No. 3:261-276

Remus, William E. (1981). Experimental Design for Analyzing Data on Games –
Or, Even the Best Statistical Methods Do Not Replace Good Experimental
Control. Simulation & Games, Vol. 12, No. 1: 3-14.

Roberts, Nancy (1976). Simulation Gaming: A Critical Review. US Department of
Health & Welfare. National Institute of Education.

Saegesser, Francois (1981). Simulation-Gaming in the Classroom. Some
Obstacles and Advantages. Simulation & Games, Vol. 12, No. 3: 281-294.

Saegesser, Francois (1984). The Introduction of Play in Schools: A Philosophical
Analysis of the Problems. Simulation & Games, Vol. 15, No. 1: 75-96.

Seidner, Constance, J. (1978). Teaching with Simulations and Games. In: Duke,
Richard E. & Seidner, Constance J. (1975)(eds.). Learning with simulations and
games. London: Sage Publications.

Sutton-Smith, Brian & Kelly-Byrne, Diana (1984). The idealization of play. In
Smith, Peter K. (ed) Play in Animals and Humans. Basil Blackwell, Inc; Oxford,
England, 1984.

Van Sickle, Ronald (1986). A Quantitative Review of Research on Instructional
Simulation Gaming: A Twenty-Year Perspective. Theory and Research in Social
Education, Vol. 14, No. 3: 245-264.

Wellington, William J. & Faria, A.J. (1996). Team Cohesion, Player Attitude, and
Performance Expectations in Simulations. Simulation & Gaming, Vol. 27, No
1:23-40.

Wentworth, D.R. & Lewis, D.R. (1973). A review of research on instructional
games and simulations in social studies education. Social Education: 432-440.

Wolfe, Joseph & Crookall, David (1998). Developing a Scientific Knowledge of
Simulation/Gaming. Simulation & Gaming, Col. 29, No. 1: 7-19.