NEW ASPECTS of APPLIED INFORMATICS, BIOMEDICAL ELECTRONICS & INFORMATICS and COMMUNICATIONS
Who Performs Better When Learning with Business Simulation Games?
A Case Study of a College General Course in Taiwan
Yu-Hui Tao1 K. C. Hung2
Department of Information Management
National University of Kaohsiung
700 Kaohsiung University Road, Nan-Tzu District
Kaohsiung, Taiwan, R.O.C.
ytao@nuk.edu.tw
2
Top-BOSS International Corp.
3F., No.19-13, Sanchong Rd., Nankang Distric
Taipei City , Taiwan, Taiwan, R.O.C.
kc@top-boss.com.tw
1
Abstract: - Previous studies on business simulation game (BSG) have concluded that performance may not be
the primary benefits for using BSG since the results are mixed. This studies aims at understanding what kind of
student profile may perform better in classroom learning with BSGs. A case study of 43 students in a general
course in Taiwan indicates that knowledge and skill do matter, and participation and tacit learning preference
contribute well to performance. However, auditory learning preference and high motivation students may not
like BSG learning in classroom. More dimensions of variables are needed to increase the explaining power of
the performance scores in regression analyses.
Key-Words: - Business simulation game, Performance, Perception, Regression, Student profile
many universities wish to culture their students
through extensive and in-depth general education in
addition to students’ majored subjects. From this
perspective, an investigation to explore the
relationship of student performance and student
profile is desirable and is the objective of this study.
In the remaining sections of this article, background
literature related to the research objective is
presented before introducing the research methods,
followed by the data analyses, discussions and
conclusions.
1 Introduction
Business simulation game (BSG) usage has reached
97.5% among the AACSB member schools in U.S.A.
more than a decade ago [2]. In early studies, the
relationship between BSG learning performance and
perception presented mixed results, and thus recently
research studies of BSG usage has been shifted to
non-performance perspectives such as attract
students’ attention and help students focus in class.
Prensky [9] claimed that games provide an
indispensable motivational condition for students’
learning, and thus students may be more interested in
choosing courses that incorporate business
simulation games in the instructional activities.
Therefore, BSG has been promoted to become
popular means for informal learning [5], and is also
being promoted for use in formal learning [8, 10].
Based on these three reasons, students’ responses to
classes using simulation games is a critical indicator
of the widespread use of simulation games in higher
education.
Extent literature focuses on business major
students. However, business operations and
management is also considered a general knowledge
for non-business major students or working
professionals who are interested in accumulated their
business knowledge and experience through learning
by doing. It is especially critical nowadays when
ISSN: 1792-460X
2 Background Literature
2.1 Business simulation games in Taiwan
Taiwan has two major business simulation game
providers,
Top-Boss
Corporation
(http://www.top-boss.com.tw)
and
Pitotech
Corporation
(http://www.pitotech.com.tw).
As
revealed in Faria [2], most simulation used in UK
universities were developed in-house while most
games used in USA were published simulations. In
Taiwan, Pitotech mainly sells imported simulations
while Top-Boss developed many games in-house.
Top-Boss offers self-developed products such as
Business Operations Simulation System (BOSS),
471
ISBN: 978-960-474-216-5
NEW ASPECTS of APPLIED INFORMATICS, BIOMEDICAL ELECTRONICS & INFORMATICS and COMMUNICATIONS
Marketing Winners, Distribution Master, Beer
Games and Retail Expert, and some imported games
such as Magnus developed by Singapore National
University. These games are as intuitive as their
names, such that BOSS is a general business
operation game, Beer game is the well known MIT
beer game (http://beergame.mit.edu/), and Retail
Expert is a convenient store game. On the other hand,
Pitotech offers imported games like Virtual Business
Management, Virtual Business Retail, and Virtual
Business Sport games, which has a smaller overall
market share in Taiwan. Other related vendors, such
as
APEX
International
Corporation
(http://www.apex.com.tw) offering Virtual Stock
Exchange game, are not appropriate candidates of
this study aiming at business management.
In recent years, Top-Boss Corporation has been
successfully promoting their products to higher
education institutes through such as free trail,
workshop, trainings camp, and sponsorships of
national business simulation game competitions.
Because these business simulation games are
team-based and competition-oriented, BOSS or other
games are incorporated in curriculum as various
group competition activities, which usually repeat
several times in a row and several days apart due to
the round of game play nature. For the students, in
each round the required parameters in many
Web-based forms need to be entered, and thus
graphical reports or visual performance data are
available for the users to increase users’ usability and
learning interests during the games. For the teachers,
they used these games throughout most of the
semester, the rationale of different expertise of
business decision-makings in the games were used to
illustrate the knowledge and applications of targeted
course subjects in class [10].
procedural knowledge, and strategic knowledge;
affective learning outcomes refer to attitudes.
However, these three categories of outcomes are not
necessary be consistent. Anderson and Lawton [1]
followed-up their previous studies and confirmed the
failure to find a relationship between simulation
performance and students’ attitude toward the
simulation. Consequently, they called for further
effort in finding the missing link between
performance and measurement items.
An interesting contrast is that Wellington and
Faria [13] confirmed that good simulation performers
are consistently good by examining high versus low
rank performers in two consecutive rounds of
simulation competition. Although no explanation
was found for this research finding, but it seems to
indicate GPA, beginning attitude toward the
simulation exercise, and team cohesiveness were not
significant differences.
Due to previous studies rarely focus on
technology perspective of BSGs, Tao et al. [11]
proposed an integrated model fusing IS theories and
education theories. In general, the empirical students’
perceptions provide adequate evidence for Taiwan’s
teachers to adopt or continue using business
simulation games.
3 Research Design
Data collection design is first described and then the
variable selection is briefly justified, followed by the
statistical methods used in this study.
3.1 Data collection design
Based on the suggestion of Tech and Murff [12], a
general course for teaching three different
complexity levels of BSGs, BOSS (a total enterprise
simulation), Retail Expert (single-player small game)
and Beer Game (multiple-player small game) from
TOP-BOSS, is offered in a general class to
undergraduate students in a university in Taiwan.
Each game is taught in a sequence of introducing the
game, practicing by simulated game competition,
writing group thought-sharing report, and gaming
competition, which was designed in reference to the
problem-solving game model [6].
The final grade calculation is divided into
attendance, individual tests, group reports and
competitions, which can be used as dependent
variables in this analysis stage. The objective is to
focus on “learning by doing” for students with or
without management background. Students learned
most of the knowledge by playing the games and
2.2 Research on business simulation games
Kiili [6] proposed a problem-based gaming model
which distinguishing the learning process into
elements
of
strategy
formation,
active
experimentation, game world observation, and
reflection. During this process, single-loop learning
is formed if the player goes from reflection to active
experimentation without forming new strategies or
double-loop learning is formed if reflections follow
by forming new strategies.
Kraiger et al. [7] classified several broad
categories of learning outcomes, including
skill-based, cognitive, and affective outcomes.
Skill-based learning outcomes address technical or
motor skills; Cognitive learning outcomes include
three subcategories of declarative knowledge,
ISSN: 1792-460X
472
ISBN: 978-960-474-216-5
NEW ASPECTS of APPLIED INFORMATICS, BIOMEDICAL ELECTRONICS & INFORMATICS and COMMUNICATIONS
performance and found out the sample groups were
significantly different in GP.
collecting needed information over the Internet.
Some tips can also be learned from the winners who
shared their wining experience at each practicing and
formal competitions, as well as from the
thought-sharing reports summarizing some necessary
knowledge, tactics and strategies for playing the
games.
A self-reported questionnaire is distributed at the
end of this class for collecting the data as described in
Section 3.2.
3.3 Statistical methods
Three different statistical methods are used in this
study: First, descriptive analysis is used to profile the
sample students; second, t-test is used to test the
performance of two groups of students with different
characteristics; third, linear regression is used to
identify the selected variables contributing to
different performance scores collected in Section 3.1
3.2 Variable selection
Students divided by gender, management major,
previous experience of management courses and
previous experience of BSG are individually tested
for their differences on performance scores. The
justification is briefly described below.
Gender has been associated with learning
performance and investigated in BSG context in
previous studies [14]. Because previous business
experience plays an important role on the BSG
performance [14] and the sample students come from
five different colleges, Management, Science,
Engineering, Society and Humanilities, and Law,
thus business major, previous experience of
management courses and previous experience of
BSG are used in this study to reflect students’
business experience.
Since BSG is like an online simulation-based
game usually playing by a group of students together,
it is natural to assume that students who like to play
online games, will influence their BSG performance.
Feinstein et al. [3] distinguished simulation from
simulators where simulators are sometimes called
iconic model with because of their visual, auditory,
and kinesthetic representations of a real system for
training purpose. Furthermore, visual, kinesthetic,
structure, learning with others, tactile, motivation and
persistence, belonging to Dunn and Dunn’s PEPS
learning style model, and were used by Hawk and
Shah [4] when comparing learning style between
undergraduate business students in American
universities and International universities.
Since the class activities are designed to be
team-work in preparing the thought-sharing reports
and two of the three game playing as well as learning
by the students themselves for management
knowledge necessary to play the games, students’
participation [15] in group discussion and
information searching off class are assumed to
contribute to the BSG performance. Wellington and
Faria [13] compared students’ simulation
ISSN: 1792-460X
4 Data Analysis
4.1 Sample profile and descriptive statistics
This general course “Business Simulation Games and
Competition” was offered to all undergraduate
students in a National University in Taiwan. Over
150 students pre-registered this course, but only fifty
one students were officially approved by the online
registration system. Forty-three of them completed
the questionnaire at the end of the class.
Tables 1 lists the means and standard devidations
(S.D.) of the perofrmance scores and measumrenet
items. A brief summary of the sample profile is
sketched as follows: 67.4% of the students were male
and only 25.6% of the students had experiences using
business simulation games in previous classes.
Students majored in Colleges of Engineering,
Management, Science, Law and Humanalities
displaying a distribution of 37.2%, 30.2%, 18.6%,
9.3% and 4.7%, respectively. In terms average scores
in previous semester, majority of the students fall into
the 80’s (55.8%) and 70s (34.9%) while very few fall
into the 50s, 60s and 90s.
4.2 Data analysis
Four individual characteristics, major, gender,
previous experience of management courses, and
previous experience of BSG, were tested against final
score. A t-test comparison of students divided by
these 4 characteristics is shown in Table 2. Even
though the means for the positive-answer (yes)
groups students are higher and with less standard
deviations, only management major and previous
experience of BSG have mild impacts on students
class scores at p=0.1 level. In other words, students
majoring in management or having BSG experiences
may potentially perform better than their
counterparts.
473
ISBN: 978-960-474-216-5
NEW ASPECTS of APPLIED INFORMATICS, BIOMEDICAL ELECTRONICS & INFORMATICS and COMMUNICATIONS
Table 1. Descriptive statistics
Item
Final score (100%)
Competition score (50%)
Sharing report score (15%)
Test score (20%)
Attendance score (15%)
Online game
Visual Learning
Auditory Learning
Kinesthetic learning
Wants structure
Tacit learning
Learning with others
High motivation
High persistence
Participating group discussion
Searching information after class
Mean/S.D.
82.74/8.19
76.77/11.18
82.17/11.80
85.97/3.29
95.60/9.68
6.37/1.94
7.40/0.93
5.97/1.40
7.29/1.13
6.30/1.46
7.05/1.17
6.60/1.29
5.63/1.31
5.67/1.24
6.37/1.00
5.35/1.45
Range
0-100
9-point Likert
scale from 1
representing
very disagree
and 9
representing
very agree
Table 2. T-test analysis between students groups for class score
Variables
Management major?
Female?
Management course before?
BSG before?
at 0.1 level
Mean/S.D.
85.23/2.83
83.86/4.60
84.47/3.33
85.09/3.05
Nyes
13
14
22
11
Mean/S.D.
81.67/9.48
80.42/12.76
81.09/10.85
81.94/9.24
Nno
30
29
21
32
t
1.847
0.977
1.395
1.683
Sig.
0.069
0.345
0.175
0.100
Table 3. Regress Analysis Results for Intention to Continuance Usage
Independent
Variable
Visual learning
Mean performance (Scores)
Sharing
-0.504***
-0.388*/-0.356*
-0.355*
-0.316*
0.319*
-
-
Kinesthetic learning
-
-
0.291
-/0.436*
-
0.562***
-/-0.469*
Wants structure
Learning with others
Attendance
Competition
Auditory learning
Tacit learning
Test
Final
-
-0.481**
-
-0.375
-/0.662*
-
-1.148***
High persistence
-
-/-0.169*
-
-0.745*
Online game
Participating group
discussion
Searching information
after class
-
-
0.440*
0.285/0.487**
High motivation
-0.341*
0.524**
0.339*
0.319
GPA last semester
F
3.779*
3.844*/3.702**
4.327*
4.620**
4.849**
Adjusted R square
0.264
0.155/0.343
0.097
0.259
0.465
***P< 0.001 ** P< 0.01 * P< 0.05 P= 0.1
ISSN: 1792-460X
474
ISBN: 978-960-474-216-5
NEW ASPECTS of APPLIED INFORMATICS, BIOMEDICAL ELECTRONICS & INFORMATICS and COMMUNICATIONS
learning-loop and reflection concepts in Kiili’s [5]
problem-based gaming model. Third, students with
auditory learning preference and high motivation
may be not used to BSG type of learning-by-doing
style. Finally, more hidden variables and dimensions
of variables are yet to be identified to increase the
explaining power of regress analysis for determing
the contributing factors of BSG performance scores
in future research.
This study is interested in learning how the
performance scores are influenced by student’s
learning styles in Taiwan context. Variables in
learning style may belong to different dimensions,
and this study adopted the following 12 variables as
seen in Table 3. Backward stepwise linear regress
was used to identify significant variables contribute
to five different performance scores.
Several observations can be derived as follows:
First, visual learning is the most significant
variable contributing negatively all but the test
performance scores. Since BSG is more of a
text-based simulation than a simulator, the feelings of
visual, auditory and kinesthetic learning do not
resemble online games or simulators. Accordingly,
students with high visual learning preference may
find it negatively associated with BSG performance
scores in most cases, except in the test score. It is
surprising to see negative influcnece on performance
scores for high motivational students, which may
imply that they are not used to this new method of
learning by doing through game competition.
Second, on the positive influnce, participating
group discussion had significant positive influence to
student performance on three out of five performance
scores. Tacit learning reflects well the nature of BSG
as a problem-based gaming involves strategies. Both
meet the original expectation of this research.
Third, test score indeed is very different from the
other scores in that three variables, online game,
searching information after class, and GPA of last
semester, that uniquely contribute to test score, but
not others. This implies that test scores may not be
appropriately evaluated by the BSG competition
activities, and vice versa. Teachers need to be very
careful in matching the course objective with its
evaluation method.
Forth, only the adjusted R-square of the
attendance score borders high level while the rest are
at low level or insignificant level. This implies, more
hidden dimensions of variables are needed to increase
the explaining power of the regression analysis on
BSG performance score.
References:
[1] Anderson, P. H. and Lawton, L. Simulation
performance and its effectiveness as a PBL
problem: A follow-up study, Developments in
Business Simulation & Experiential Exercises,
Vol. 34, 2007, pp. 43-49.
[2] Faria, A. J. and M. Schumacher. Business
Simulation games: Current usage a ten year
update, Developments in Business Simulation &
Experiential Exercises, Vol. 11, 1996, pp.
220-225.
[3] Feinstein, A. H., Mann, S. and Corsun, D. L.
Charting the experiential territory: Clarifying
definitions and uses of computer simulation,
games, and role play. Journal of Management
Development, Vol. 21, No. 10, 2002, pp.
732-744.
[4] Hawk, F. F. and Shah, A. J. Using learning style
instruments to enhance student learning,
Decision Sciences Journal of Innovative
Education, Vol. 5, No. 1, 2007, pp. 1-19.
[5] Kapp, K. Gadgest, games and gizmos: Informal
learning
at
Nick.com.
http://karlkap.blogspot.com, 2006.
[6] Kiili, K.
Foundation for problem-based
gaming, British Journal of Educational
Technology, Vol. 38, No. 3, 2007, pp. 394-404.
[7] Kraiger, K., Ford, J. K., and Salas, E.
Application of cognitive, skill-based, and
affective theories of learning outcomes to new
methods of training evaluation. Journal of
Applied Psychology, Vol. 78, No. 2, pp.
311-328
[8] Lim, C. P. Spirit of the game: Empowering
students as designers in schools. British Journal
of Educational Technology, Vol. 39, No. 6,
2008, pp. 996-1003.
[9] Prensky, M. Digital game-based learning.
Computer in Entertainment, Vol. 1, No. 1, 2003,
pp. 21-21.
[10] Prensky, M. Students as designers and creators
of educational computer games: Who else?
British Journal of Educational Technology, Vol.
39, No. 6, 2008, pp. 1004-1019.
5 Conclusion
We conclude several findings on who may perform
better with BSG learning activities based on the
above analysis and discussion. First, certain
knowledge (management major) and skill (previous
BSG experience) do matter in terms of student
performance. Second, participating group discussion
and tacit learning are positive factors in influencing
student
performance, which
matches
the
ISSN: 1792-460X
475
ISBN: 978-960-474-216-5
NEW ASPECTS of APPLIED INFORMATICS, BIOMEDICAL ELECTRONICS & INFORMATICS and COMMUNICATIONS
[11] Tao, Y.-H., Cheng, C.-J. and Sun, S.-Y., What
influences college students to continue using
business simulation games? The Taiwan
experience, Computers & Education, Vol. 53,
No. 3, 2009, pp. 929-939
[12] Tech, R. and Murff, E. R. T. Are the business
simulations
we
play
too
complex,
Developments in Business Simulation &
Experiential Exercises, Vol. 35, 2008, pp.
205-211.
[13] Wellington,W. J. and Faria, A. J. An
investigation of the relationship between
simulation play, performance level and recency
of play on exam scores, Developments in
Business Simulation & Experiential Exercises,
Vol. 18, 1991, pp. 111-114.
[14] Wellington, W. J. and Faria, A. J. Are good
simulation performers consistently good,
Developments in Business Simulation &
ExperientExercises, Vol. 22, 1995, pp.5-11.
[15] Whiteley, T. R. and Faria, A. J.. A study of the
relationship between student final exam
performance and simulation game participation.
Simulation & Gaming, Vol. 20, 1990, pp. 44-64.
ISSN: 1792-460X
476
ISBN: 978-960-474-216-5