Broadening Educational Game Design
using the World Wide Web
Doctoral Dissertation submitted to the
Faculty of Informatics of the Università della Svizzera Italiana
in partial fulfillment of the requirements for the degree of
Doctor of Philosophy
presented by
Navid Ahmadi
under the supervision of
Prof. Mehdi Jazayeri
December 2012
Dissertation Committee
Prof. Fabio Crestani
Prof. Marc Langheinrich
Università della Svizzera Italiana, Switzerland
Università della Svizzera Italiana, Switzerland
Prof. Alexander Repenning
Prof. Mary Beth Rosson
University of Colorado at Boulder, USA
Pennsylvania State University, USA
Dissertation accepted on 20 December 2012
Prof. Mehdi Jazayeri
Research Advisor
Università della Svizzera Italiana, Switzerland
Prof. Antonio Carzaniga
PhD Program Director
i
I certify that except where due acknowledgement has been given, the work presented in this thesis is that of the author alone; the work has not been submitted
previously, in whole or in part, to qualify for any other academic award; and the content of the thesis is the result of work which has been carried out since the official
commencement date of the approved research program.
Navid Ahmadi
Lugano, 20 December 2012
ii
To my parents...
iii
iv
Abstract
Educational game design has been increasingly used for teaching computational thinking skills from kindergarten to college. Learners deal with concepts such as objects,
instances, behavior, and interaction and and gain programming skills while they are
engaged in creating a video game. We identify two limitations in current educational
game design environments. First, the learning activities are tied to the presence of
a teacher who helps the learner to solve problems that require computational thinking. These activities often take place in an educational setting such as a classroom or
after-school computer club. The learning opportunities are constrained to face-to-face
communication in a physical location. Therefore, those with no access to an educational setting miss the learning opportunities. Second, Computer Science courses have
been perceived as being ‘hard’, ‘boring’, and ‘asocial’ among learners. Educational
game design environments have employed end-user development techniques to cope
with the ‘hardness’ and put these techniques into the context of creating a game to
cope with ‘boredom’. However, the ‘asocial’ face of computing has been addressed far
less.
The World Wide Web has become a dominant platform for computer-supported
cooperative work and social media. Cyberlearning platforms such as Massive Open
Online Courses are increasingly employing the Web to deliver education to hundreds
of thousands of learners across the world. These platforms rely on the participative culture of the Web, user-created content, social networking applications, and other social
media to achieve their educational goals. However, game design environments have
not leveraged such rich media to go beyond the classroom environment. The Web can
be employed as a cyberlearning platform for educational game design. Such a platform
broadens the scope of game design practices, allowing participation to those without
access to a physical educational setting. Learners rely on peers and communities to
gain computational thinking skills. Ultimately, such a platform can change the asocial
face of computing.
In this thesis, I explore the feasibility of employing the Web as a cyberlearning
platform for educational game design, in two steps: 1) by taking the game design activities to the Web. I have created an open-Web educational game design environment
and demonstrated that the Web is a suitable hosting platform for game design activities
including creating game objects, programming them, and designing the game scene,
v
vi
all with a visual interface. 2) by transferring the educational practices from the classroom to the Web. I have developed a cooperative Website around the game design
environment, which allows users to share and explore games and create a community
around their game design activities.
My experiments with a group of novices validates their success in using the cyberlearning platform to gain computational thinking skills during the game design activities, while relying on the cooperative features of the Website to gain knowledge and
learn computational concepts.
Acknowledgements
This research has been a long journey for me. It is hard to acknowledge everybody
who inspired, affected, or helped me in this work. I attempt my best and apologize for
those whom I omit.
Foremost, I would like to express my highest gratitude to my advisor, Mehdi Jazayeri, for encouraging and supporting me throughout this work. He constantly taught
me the more objective way of thinking, and patiently let me find my own way. It has
been a privilege for me to be your student.
I extend my sincere gratitude to Alexander Repenning, who introduced me to the
end-user programming and its educational values. His great care about computer science education in schools has highly affected my research. I am thankful to Nadia
Repenning, who made me feel home during my stay in Boulder, CO. I am grateful to
the members of Scalable Game Design project, in particular, Kyu Han Koh, Vicki Bennett, Burke Taft, and Sandra Wilder for providing me with opportunities to evaluate
my system with novices.
I am sincerely grateful to Gerhard Fischer for hosting me in the Center for Life-Long
Learning and Design (L3 D) during my visit to the University of Colorado at Boulder.
I learned so much about human aspects of computing throughout many stimulating
discussions and inspiring meeting in L3 D. I would like to thank the members of L3 D, in
particular, Hal Eden and Holger Dick, for their friendly support.
I am thankful to Monica Landoni, who helped me evaluate AgentWeb with the
school teachers and students in Ticino, Switzerland. Without her great care and support, this would not have been possible.
I would like to thank my office-mates, formerly Francesco Lelli and Sasa Nesic,
and lately Domenico Bianculli, who inspired me by their commitment to research and
brought fun to our office in Lugano. I would like to thank Julia Eberle, Zhu Li, and Ivan
Vaghi for several stimulating discussions that we had on software design, information
technology, and education during my stay in Boulder.
I would like to thank my parents, Manouchehr and Ozra, for their never-ending
love, encouragement, and support. I wish the words were not so limited to tell you
how proud I am to be your son.
Finally, I wish to thank my beloved girlfriend, Sadaf, for her love, care, and patience, from thousands of miles away. Life is more colorful when I think about you.
vii
viii
Contents
Contents
ix
List of Figures
xiii
List of Tables
xv
1 Introduction
1.1 Challenges in Educational Game Design . . .
1.2 Cyberlearning as a Solution Framework . . .
1.3 World-Wide Web as a Cyberlearning Platform
1.4 Thesis Statement . . . . . . . . . . . . . . . . .
1.5 Research Method . . . . . . . . . . . . . . . . .
1.6 Thesis Structure . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
2 Related Work
2.1 End-user Development . . . . . . . . . . . . . . . . . . . . . . .
2.1.1 End-User Programming . . . . . . . . . . . . . . . . . .
2.1.2 Collaboration practices among end users . . . . . . .
2.1.3 Collaborative and Social Software Engineering . . .
2.2 Educational Game Design . . . . . . . . . . . . . . . . . . . . .
2.2.1 Game-Based Learning . . . . . . . . . . . . . . . . . . .
2.2.2 Educational Game Design Environments . . . . . . .
2.2.3 State-of-the-art of support for collaboration in GDEs
2.3 Computational Thinking . . . . . . . . . . . . . . . . . . . . . .
2.3.1 Computational Thinking Patterns . . . . . . . . . . . .
2.4 World-Wide Web . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.4.1 Participative Web . . . . . . . . . . . . . . . . . . . . . .
2.4.2 Web 2.0 for Education . . . . . . . . . . . . . . . . . .
2.5 Web as a runtime platform . . . . . . . . . . . . . . . . . . . .
2.5.1 Programmable Web and Mashups . . . . . . . . . . .
2.5.2 RIA Platforms vs. Open Web for Games . . . . . . . .
2.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
ix
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1
3
4
5
6
6
7
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
9
9
10
13
14
15
15
16
24
25
26
27
27
28
29
29
30
31
x
Contents
3 Building an Open-Web Game Design Environment
3.1 System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.1.1 Choice of game design model and support for end users .
3.1.2 Choice of collaboration platform . . . . . . . . . . . . . . .
3.1.3 Choice of Web technology . . . . . . . . . . . . . . . . . . .
3.2 Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3 Feasibility Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3.1 Ristretto Mobile: an AgentSheets to JavaScript Compiler
3.3.2 Web-based Visual Programming of Physical Devices . . .
3.4 User Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.5 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.6 Visual Programming Language . . . . . . . . . . . . . . . . . . . . .
3.6.1 Language Specification . . . . . . . . . . . . . . . . . . . . .
3.7 Client-side Execution . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.8 Performance Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.8.1 Code Compilation . . . . . . . . . . . . . . . . . . . . . . . .
3.8.2 Differential Scene Rendering . . . . . . . . . . . . . . . . .
3.9 Performance Evaluation . . . . . . . . . . . . . . . . . . . . . . . . .
3.10 Usability Testing of the Game Design Environment . . . . . . . . .
3.11 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4 Transition to the Web as a Cyberlearning Platform
4.1 A process to support design and computational thinking . . . . .
4.1.1 Identified Computational and Design Thinking Activities
4.1.2 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.1.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2 Comments from Teachers: Towards Smarter GDEs . . . . . . . . .
4.2.1 Examples, Tutorials . . . . . . . . . . . . . . . . . . . . . . .
4.2.2 Computational Thinking Issues . . . . . . . . . . . . . . . .
4.2.3 Design Thinking Issues . . . . . . . . . . . . . . . . . . . . .
4.2.4 Software Engineering Process . . . . . . . . . . . . . . . . .
4.3 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5 Building a Cooperative Game Design WebSite
5.1 Issues with existing models of cooperation .
5.2 A Vision for Unobtrusive Cooperation . . . .
5.3 Implementation . . . . . . . . . . . . . . . . .
5.4 Content Development . . . . . . . . . . . . .
5.4.1 Video Tutorials . . . . . . . . . . . . .
5.4.2 Forum . . . . . . . . . . . . . . . . . .
5.4.3 Showcase . . . . . . . . . . . . . . . .
5.5 Usability Evaluation of the Website . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
33
34
34
34
36
36
37
37
40
42
43
46
46
47
49
50
51
51
56
57
.
.
.
.
.
.
.
.
.
.
59
60
61
62
64
65
66
67
68
69
70
.
.
.
.
.
.
.
.
71
72
73
74
76
76
77
77
77
xi
Contents
5.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6 Experimenting with Cooperation
6.1 Research Questions . . . . . . . . . . . . . . . .
6.2 Research Method Overview . . . . . . . . . . .
6.3 Data Collection Methods . . . . . . . . . . . . .
6.3.1 Computer Logs . . . . . . . . . . . . . .
6.4 Procedure . . . . . . . . . . . . . . . . . . . . . .
6.4.1 Sample . . . . . . . . . . . . . . . . . . .
6.4.2 Pre Test . . . . . . . . . . . . . . . . . . .
6.4.3 Game Design Session . . . . . . . . . .
6.4.4 Post test . . . . . . . . . . . . . . . . . .
6.5 Results . . . . . . . . . . . . . . . . . . . . . . . .
6.5.1 Task completion and quiz assessment
6.5.2 Cooperation . . . . . . . . . . . . . . . .
6.5.3 Details of game design activities . . .
6.6 Conclusions . . . . . . . . . . . . . . . . . . . . .
79
.
.
.
.
.
.
.
.
.
.
.
.
.
.
81
82
82
82
83
84
84
84
85
89
89
90
90
92
93
7 Conclusions
7.1 Research Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.2 Open Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.3 Future Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
95
95
96
97
A AgentWeb’s Visual Programming Language Specification
99
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
B Usability Evaluation Form
B.1 Intellectual task . . . . . . . . . . . . . . . . . . .
B.2 User Interface Task . . . . . . . . . . . . . . . . .
B.3 Computational Thinking Task . . . . . . . . . . .
B.3.1 Matching with the sheet . . . . . . . . .
B.3.2 Programming . . . . . . . . . . . . . . . .
B.4 Computation Thinking Pattern Matching Sheet
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
103
103
104
105
105
105
106
C Programming Tutorial Manuscript
C.1 List of video tutorials . . . . . . . . . . . .
C.2 Introducing AgentWeb . . . . . . . . . . .
C.3 Creating a simple game with AgentWeb
C.4 Using the Depiction Editor . . . . . . . .
C.5 Drawing the game scene . . . . . . . . . .
C.6 Programming . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
109
109
109
111
111
112
112
D Evaluation Forms
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
117
xii
Contents
Figures
2.1 Research Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10
2.2 Cost of learning versus Scope of EUD tools, adapted from [39] . . . . . .
11
2.3 AgentSheets Game Design Environment . . . . . . . . . . . . . . . . . . . .
17
2.4 Stagecast Creator Game Design Environment . . . . . . . . . . . . . . . . .
18
2.5 Squeak Etoys Game Design Environment . . . . . . . . . . . . . . . . . . . .
19
2.6 Alice 3D Game Design Environment . . . . . . . . . . . . . . . . . . . . . . .
20
2.7 Game Maker Game Design Environment . . . . . . . . . . . . . . . . . . . .
21
2.8 Scratch Game Design Environment . . . . . . . . . . . . . . . . . . . . . . .
22
2.9 Greenfoot Game Design Environment . . . . . . . . . . . . . . . . . . . . . .
23
2.10 Kodu Game Design Environment . . . . . . . . . . . . . . . . . . . . . . . . .
24
2.11 AgentCubes Game Design Environment . . . . . . . . . . . . . . . . . . . .
25
3.1 Ristretto Mobile compiles AgentSheets applications to HTML5 to be executed inside the Web browser. . . . . . . . . . . . . . . . . . . . . . . . . . .
40
3.2 A first-generation iPhone running a Sokoban game inside Mobile Safari. The game was created by AgentSheets and compiled to HTML5 by
Ristretto Mobile. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
41
3.3 JavaScript-based visual rule-based programming language was created
to rapidly compose the behavior of Lego MindStorms NXT. . . . . . . . .
42
3.4 The AgentWeb IDE, used by novice programmers to develop open-Web
games inside the browser. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
44
3.5 The overview of AgentWeb architecture: life-cycle of agents, user interaction with the system, and flow of operations during the execution
process are shown. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
45
3.6 The control flow of a method in an agent program. . . . . . . . . . . . . .
47
3.7 Main conditions and actions of AgentWeb . . . . . . . . . . . . . . . . . . .
48
3.8 Runtime system components and their relationship. . . . . . . . . . . . . .
49
xiii
xiv
Figures
3.9 On the left side (a,b,c,d), the display rate is measured from execution of
games in a number of browsers. Each browser has been benchmarked
with four different configurations. Configurations are ordered from the
slowest with no optimization (N) on the left to the fastest with code
compilation and differential scene rendering on the right (CD). Evaluations with the result of less than 3 frames per second are not visible; On the right side (e,f), the performance of the same browsers are
measured using two browser benchmarking suites: e) In SunSpider, the
faster browser has the lower benchmarking time. f) In V8, the faster
browser has the higher score. . . . . . . . . . . . . . . . . . . . . . . . . . .
3.10 A sample usage pattern automatically recorded from end user’s interaction with Web-based AgentSheets . X axis is the passage of time in
minutes. Y axis is the section in which user has spent the time. The time
periods of less than 10 seconds are considered as noise and removed. . .
4.1 Evaluated game design process combines the computational thinking
tasks with tool usage tasks. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2 Mean of teachers’ rating of research questions based on cognitive jogthrough’s scaling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
55
57
63
65
5.1 Mode of participants’ rating based on the likert scale. . . . . . . . . . . . .
78
6.1 The participant receives the description of tasks from inside the game
design environment. The logging system records the flow of completing
the tasks and the time spent on each task. . . . . . . . . . . . . . . . . . . .
86
B.1 Computational Thinking Patterns, Page 1 . . . . . . . . . . . . . . . . . . . 107
B.2 Computational Thinking Patterns, Page 2 . . . . . . . . . . . . . . . . . . . 108
Tables
3.1 Challenges of creating an educational game design environment using
open-Web technologies and corresponding proposed solutions. . . . . . .
3.2 Evaluated Games and Their Properties . . . . . . . . . . . . . . . . . . . . .
38
53
5.1 Comparison of Game Design 1.0 and Game Design 2.0 styles . . . . . . .
75
6.1 Results of the experiment, divided by the tasks. . . . . . . . . . . . . . . .
90
A.1 Visual Data Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
A.2 List of Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
A.3 List of Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
C.1 List of video tutorials available on AgentWeb Website . . . . . . . . . . . . 109
C.2 List of video tutorials of conditions and actions that are embedded into
AgentWeb’s programming environment . . . . . . . . . . . . . . . . . . . . . 110
xv
xvi
Tables
Chapter 1
Introduction
Computer science is a growing field in both academia and industry. With the ubiquity
of digitized information around us, computational thinking is a must-have skill in the
21st century [147]. The growth of number of end-users who are engaged in programming activities proves the increasing need to computational solutions [119]. However,
gaining computational thinking skills is a challenge to novices.
To learn computer science concepts, often students are engaged in programming
activities which let them gradually gain the expected skills. Papert and others created
Logo, the first educational programming language to support teaching programming
and CS concepts in 1967 [94]. Since then, many programming languages and environments have been created and used in schools and universities. These environments
follow constructivism [14] and constructionism [58] learning theories and practices
and they have been influential on how programming languages are taught.
Since early 90s, educational programming environments started to form the programming practices around game design activities, i.e., becoming an ‘Educational
Game Design Environment’. In this thesis I use the acronym ‘GDE’ to refer to these
environments. In such an environment, the learner plays the role of a game designer/programmer which involves designing and programming graphical objects that interact with each other and with the player in a game-like scene. Educational Game
Design may point to two different yet related definitions:
1. Teachers use educational games during teaching different courses. One field of
research and practice focuses on how to design these educational games to best
convey the teaching material while keeping the learner engaged [85].
2. In the course of teaching, in particular in Science, Technology, Engineering, and
Math (STEM) classes, the teacher engages her students in creating a video game
or scientific simulation during which they learn the expected teaching material1 .
1
In several cases, students are engaged in creating interactive artifacts such as animations and stories,
rather than a full game. For the sake of simplicity for the reader, I consider all of the interactive artifacts
as games.
1
2
The first definition is a relevant but secondary topic to this dissertation. The focus
of this dissertation is the second definition. I shall use the term ’Educational Game
Design’ hereafter and throughout this dissertation to point to the second definition.
The usual audience of educational game design environments are users with no or
little programming skills who are known as novice programmers [61], or in short
novices, who also are known as end users if they are not interested in programming
per se [87]. In this thesis, I will use these terms interchangeably.
Educational games and simulations are being increasingly adopted in public schools
[54]. Students explore computer science and software engineering [25][121] concepts
such as objects, agents, instances, behavior, abstraction, decomposition, and interaction through creating game characters, programming them, enacting and observing
their behavior in the game scene. AgentSheets [101], Alice [27], Squeak Etoys [59],
GameMaker [93], and Scratch [112] are examples of such educational game design environments. Kelleher and Pausch provide a comprehensive survey on these languages,
their educational purposes, and their programming paradigm [61].
In an emerging view on the general benefits of computing knowledge, whether the
students eventually pursue computer science or any other discipline, the term ‘computational thinking’ has gained much attention [147]. Recently, game design courses
with their focus on computational thinking have been designed, taught and reported.
Accordingly, educational game design environments are used to help students gain
computational thinking skills. The wider impact of gaining computational thinking
skills, in addition to just learning how to program, have only recently been recognized.
Educational game design environments have been around for more than two decades.
As games have become more and more popular in education, they have been often used
in STEM education [54],[128] [103],[93]. They have matured enough to let us know
their characteristics:
• GDEs use a set of end-user programming tools and techniques to support novices.
These techniques include visual languages, domain specific languages, and programming by example.
• Existing GDEs are solitary desktop-based applications. They have no or limited
support for sharing and collaboration. The extent of collaboration has not gone
beyond a supplementary Website which is used as a forum for communication
and for sharing the games.
• Game design activities mostly take place in classrooms and after-school clubs,
i.e., a physical location in the presence of a teacher.
• Most of the existing GDEs are available free, or at least have a free trial version,
on Windows and Mac OS operating systems.
3
1.1 Challenges in Educational Game Design
1.1 Challenges in Educational Game Design
There are a number of challenges regarding the scope and extent of use of existing
game design environments in education. Here I review some of the well-known challenges that are relevant to this theis.
With respect to the use of game design environments in education, we can divide
the users into two groups: those with access to a teacher, and those with no access to
a teacher. For those who have a teacher, the majority of teaching takes place in the
context of a school course or after-school computer club. Even when GDEs are used
in the schools, the outcomes are not clear. A comprehensive study on the computer
science education across the United States [146] reveals:
This study finds that current federal, state, and local government policies underpinning the K–12 education system are deeply confused, conflicted, or inadequate to teach engaging computer science as an academic
subject. Quality instruction always depends on knowledgeable and wellprepared teachers, on instructional materials that are engaging and carefully developed to enable student learning, and adequate resources and
infrastructure to support teachers and student learning. These goals must
be supported by a policy framework that sustains teacher development;
certification and continuing education; appropriate curriculum development; and student access and interest. When it comes to computer science
education, this framework is failing.
Lack of an assessment framework contributes to the failure of building a standard
and promising curriculum [146] [141]: “Assessments for computer science education
are virtually non-existent." Furthermore, game design practices are tightly bound to
the presence of a teacher, which mainly takes place in school and after-school settings
in which a teacher is present. Despite free availability of educational game design software for download, those with no access to an educational context are hardly interested in or capable of challenging with computational abstractions and programming
concepts on their own.
Another challenge is the lack of leveraging social interactions among the learners.
Lack of support for collaboration among learners in most of the educational contexts
has undermined computer science education, whether it has been due to the structure
of the course or the lack of tools for collaboration. For instance, computing fields are
streotyped as being asocial, causing low participation of women in the field [1] [113].
In contrast to the misperception of computer science as an asocial area, it has been
shown in several cases that enabling collaboration among students of an introductory
computer science course has led to better educational outcomes. For instance, pair
programming has improved learning outcomes in teaching computer science [144].
In another example, employing peer learning has led to better results among students
4
1.2 Cyberlearning as a Solution Framework
[145]. Cooperative work among novices is a well-studied phenomenon. End users
often seek help from more skilled peers in the workplace [87]. When it comes to using
GDEs, support for communication and collaboration among the learners is missing.
Existing game design environments are solitary desktop-based applications. They have
no or very limited support for sharing and collaboration. Kelleher laments the lack
of social support and considers it as the sociological barrier to programming among
novices [61]. Even worse, with the growth of social media in the past decade, game
design activities have not employed socio-technical systems for integrating into the
learner’s lifestyle, being left as an activity to be done, if ever, in the school.
These challenges raise the following questions:
• Can we broaden computer-science education beyond the classroom, for instance
through a Massive Open Online Course (MOOCs) [71] [65], to provide equal
educational opportunity to those with no access to a formal educational setting
or an appointed teacher?
• Can we leverage socio-technical systems in computer science education to change
the asocial face of computing and integrate the game design practices into the
social lifestyle of students?
This thesis reflects my efforts to answer these questions.
1.2 Cyberlearning as a Solution Framework
Cyberlearning is a term which refers to educational practices that leverage networked
platforms. The opportunities provided by networked platforms are studied and suggested for learning [19]. I have explored the cyberlearning opportunities to address
the aforementioned challenges. Relying on networked technologies opens a whole new
dimension of possibilities for learning:
• Using a virtual learning environment, we can transcend some of the limitations
associated with time and space. Learning can take place anywhere/anytime.
• In a networked environment, the lack of a formal teacher can be compensated
by building a community of practitioners which help each other to gain expertise. Moreover, teachers can use the environment as a supporting tool in their
classroom, or extend their classroom beyond a physical location in their school.
• In a networked environment, there will be opportunities to integrate educational
game design into students’ and young people’s daily use of social technology, enabling them to seamlessly work on their projects at home and school. Cooperative and social game design scenarios can change the asocial face of computer
science, leading to an increase in participation of women in computer science
[130].
5
1.3 World-Wide Web as a Cyberlearning Platform
To narrow down my research, I raise two fundamental questions, to be answered
in this thesis:
• How can a networked platform realize these goals?
• Would it be possible to replicate computer science education in a networked
learning environment with no formal teacher?
1.3 World-Wide Web as a Cyberlearning Platform
The Web has brought us an information universe which has changed how people live
and work [16]. During the last decade, the Web has evolved from an information
platform to a social platform for sharing, collaboration, participation, and education
[136]. Above all the impact that the Web has made in many aspects of our lives, it has
remained open and non-proprietary.
The Web has evolved how software is developed and delivered to end users [92].
Moreover, the Web has become the main gateway to access the software as a service
[133]. I believe that these benefits have the potential to address all the challenges that
were discussed earlier regarding computer science education
• The Web is accessible from all devices and platforms for end users.
• A Web-based platform can reach its audience globally. At the moment, there
are examples of Web-based educational platforms that have succeeded to reach
hundreds of thousands of students per course.
• In the software-as-a-service paradigm, users enjoy software updates as soon as
they are available.
• When game design takes place “in the cloud", monitoring and collecting game
design patterns is greatly facilitated. Mining such data can lead to developing
comprehensive assessment frameworks. To the best of my knowledge, to date
no information on user interaction with an education game design environment
has been collected.
• Assuming that an assessment tool is developed, it can quickly reach the learners,
track learners’ progress and provide immediate feedback on their progress.
With the success of Websites that allow virtually any user to create content and
share it globally with everyone on the Internet, the concept of Web 2.0 [91] and its
participative culture [136] was formed. Soon, the potential of the participative Web
applications to bring education to the masses was recognized [17]. The advent of
participative Web has engendered social learning environments [8] and E-learning 2.0
[19]. By taking educational game design to the Web, we can leverage the participative
and social culture of the Web to:
6
1.4 Thesis Statement
• Shift the game design towards a community-based practice which relies on cooperation, collaboration, and peer learning.
• Integrate the game design activities into the users’ social life style through online
social media and social networking applications
The intention of this thesis is to leverage the opportunities provided by the Web,
its participative culture, and its service-oriented software development and delivery
to the benefits of educational game design. However, no great benefit comes without challenges. The original Web was not intended to be used for building game and
media-rich applications. Therefore, third-party proprietary technologies such as Flash
[9] was used to deliver rich and interactive media to the Web. For educational purposes, it has been my intention to avoid any proprietary software. I rely on open-Web
technologies that are standardized by W3C consortium to deliver a rich-media game
design platform to the Web. As software, an educational game design environment
consists of components that make it challenging to build using open-Web technologies,
going beyond today’s practices and patterns for building open-Web applications. Such
components include a visual domain-specific programming language and its compiler,
a graphical scene authoring tool, a game engine and its graphic rendering engine, and
putting them all together as an integrated development environment (IDE) suitable
for novice programmers. A major part of this research has involved exploring how
to leverage open-Web technologies to build a media-rich environment beyond existing
Web application practices.
1.4 Thesis Statement
In this thesis, I verify and validate the following statements:
• The open Web is a suitable hosting platform for creating and executing educational
games in a visual interface with the standard end-user development techniques that
are used to lower the engagement barriers for novices. We can replicate game design
practices right on the Web.
• We can replicate the classroom educational practices for delivering computational
thinking skills in a Web-based game design platform.
These two assertions establish the foundations needed to scale up computer science
education beyond the classroom environment.
1.5 Research Method
To verify the assertions, I have developed a fully Web-based educational game design
environment, called AgentWeb, that provides instant access to the design and implementation of the games, enabling students to share and cooperate on exploring the
7
1.6 Thesis Structure
internals of the games, including their design and program. During the development
process, I have overcome several engineering challenges including software architecture, performance, deployment, and usability.
To enable educational practices between peers and compensate for the lack of a
teacher, I have designed and developed a cooperative Website which hosts the game
design environment.
I have conducted a number of studies on AgentWeb with novices, to:
• Improve the usability of the game design environment.
• Understand the requirements of a cooperative game design Website.
• Perform educational practices and understand the function of cooperation tools
in the learning process.
1.6 Thesis Structure
The remainder of this dissertation is organized as follows.
• In Chapter 2, I describe the related work in those fields that together have created a foundation for this dissertation, including end-user development, computational thinking, educational game design, Web technologies, participative Web
and learning 2.0.
• In Chapter 3, I describe the development of AgentWeb. The chapter includes my
rationale of using open-Web standards as the development technology, AgentWeb’s architecture, it’s user interface, employed end-user programming techniques including visual programming and interactive programming, its underlying compiler and runtime system, its performance issues and employed optimization techniques, and initial usability testing of the environment.
• In Chapter 4, I discuss the fundamental differences between educational game
design in a class-room activity led by a teacher and a cyberlearning game design
platform based on peer learning mechanics. This chapter includes an evaluation
with four high-school Informatics teachers who contributed to understanding
the requirements of the new platform and my reflection on their comments to
improve the software to support learning activities in the absence of a teacher.
• In Chapter 5, I discuss the development of a cooperative Website around AgentWeb game design environment. The Website incorporates the computer-supported
cooperative tools to enable sharing and collaboration among game designers.
The cooperation tools are designed based on the Web 2.0 and social networking
practices and follow the comments of teachers described in Chapter 4.
8
1.6 Thesis Structure
• In Chapter 6, I report on an experiment that I conducted in order to measure the
performance of novices while solving computational thinking problems and the
function of cooperation features during the experiment.
• In Chapter 7, I reflect on the results of the research I have conducted, and plan
a future roadmap for this work.
• In Appendices, I attach the details of the programming language of AgentWeb
and evaluation forms that were used in the evaluation parts of this thesis.
Chapter 2
Related Work
The work presented in this thesis focuses on supporting novices through tools and
techniques that have been developed and studied in the area of end-user development
(Section 2.1). Such techniques have been used to develop educational game design
environments (Sections 2.2) and for teaching computer science concepts and computational thinking (Section 2.3). I have relied on the World-Wide Web and its participative culture to build a cyberlearning environment for educational game design
(Section 2.4). Accordingly, I have employed the Web as an application development
and runtime platform for novices to build and execute interactive applications, beyond
its initial purpose as an information sharing platform (Section 2.5). Figure 2.1 shows
the general scope of this thesis.
2.1 End-user Development
As end users continue to use a software, new requirements and use-cases appear. To
address the needs of end users, one approach is to enable them to customize a software
according to their needs, which is known as end-user development (EUD). Lieberman
et al. [69] define end-user development as "a set of methods, techniques, and tools
that allow users of software systems, who are acting as non-professional software developers, at some point to create, modify or extend a software artefact." Several types
of development activities that are carried out by end users have been identified [83]:
• Customization allows users to modify the appearance of presentation objects, or
edit their attribute values by selecting among a set of predefined configuration
options.
• Integration goes beyond customization by allowing users to add new functionality to the software without accessing the underlying implementation.
• Extension allows users to improve the functionality of the software which may
involve modifying or restructuring the underlying implementation.
9
10
2.1 End-user Development
Computer-Science Education
World-Wide Web
Software
Engineering
Figure 2.1. Research Space
Often, user requirements grow as they continue to use a software. Soon, customization will not be enough to respond to user expectations. End users are presumably
non-programmers. Therefore, designers consider a ‘gentle slope’ of increasing complexity to enable end users to move from customizing the software towards extending
it.
2.1.1 End-User Programming
When end users reach to the point that they require to modify a software or add new
functionality, they will need programming skills which they most-probably do not have.
End-user programming (EUP) tools and techniques have been developed to empower
end users who are not interested in programming per se but need to perform computational tasks [87].
EUP tools can be categorized on two dimensions: generality and learning cost. Figure 2.2 shows how some of existing EUP tools are categorized along these dimensions
[39]. The ideal EUP tool is the one that is more general and has lower learning cost.
These tools are also known as high-ceiling, low-threshold tools [86]: “The threshold is
how difficult it is to learn how to use the system, and the ceiling is how much can be
done using the system."
Considering the growing number of end-user programmers [119], there is no doubt
entering parameters in a form-filling dialogue. Closer to the higher scope boundary are Macro
languages that extend the office style applications, e.g. formulae for Excel spreadsheets, and
database query languages. Finally the high scope, low cost cell is the EUD ideal; although as yet
it is largely unattained. The current state-of-the-art EUD environments provide graphical worlds
to create
of instructing
agents with
11 programmable agents. These still impose a learning burden
2.1 End-user
Development
condition-action rules, and designing agent models.
Cost of learning
High
Low
EUD
ideal
JAVA
C++
Current EUD envs
Agentsheets
Alice
High
JAVA Script
VB Script
Excel macros
Scope
Low
Domain engineering
languages
SDL
Hardware design
Office Applications
Report writers
Query screen
builders
Domain-specific
languages
Customisation
Adaptation
Figure 1. Cost-scope trade-offs in EUD tools
Figure 2.2. Cost of learning versus Scope of EUD tools, adapted from [39]
Active EUD environments attempt to infer programs as instructions from user manipulations of
agent worlds. The graphical agent worlds still have to be designed but, once present,
that some of the programs written by end users affects our daily lives. For instance,
programming by example [Lieberman, 2001] can infer instructions from the users’ actions, e.g. in
in spreadsheets
arebumping
used extensively
statistical
and accounting
a robotformulas
game thewritten
user demonstrates
an agent
into a wallfor
followed
by reversing
two steps
purposes.
Therefore,
on end-user
programrule
becomes
important. End-user
and changing
direction.
The dependability
system infers the
condition-action
of detect-a-collision
followed
Software
Engineering
is
a
field
that
studies
tools
and
techniques
for
end-users
beyond
by the appropriate reverse-and-change-direction response. This approach reduces learning by
programming,
including testing
debugging
semi-automatic
rule acquisition
but the and
downside
is that[22]
the [63].
learning system can make mistakes.
Learning styles
from
complete
inference
instruction,
where
theare
system
In therange
rest of
thismore
section
I describe
sometoofdirect
the EUP
techniques
that
usedlearns
in
only when
given
a
command.
Directed
instruction
requires
the
user
to
anticipate
all
the
possible
this thesis.
rules and learning situations, while the complete inference approach is limited by the system’s
domain knowledge. Developing the model is the hard part and therein lies the real challenge for
end-user
design:
abstract conceptual thinking. Complex domains require sophisticated analysis
Direct
Manipulation
and modelling skills, so programming is only part of an end-user developer’s needs.
Direct manipulation interfaces represent the application domain objects continuously
so that
users tools
can interact
with the
as they
would dopowerful
in the real
world to
The goal
for EUD
is to reduce
the objects
learningnaturally
burden while
providing
facilities
address[122].
a wideDirect
rangemanipulation
of problems. is
Given
that some
learningthe
burden
willprogrammatically.
always be present,For
tools
opposed
to modifying
objects
need toinstance,
motivatea their
users.
We
propose
a
meta-design
approach
[Fischer
&
Giaccardi,
2004],
graphical image editor such as PhotoShop allows users to choose a shape
and draw it using a mouse, while in AutoCAD graphical objects can be created using a
command line.
Graphical User Interfaces (GUIs) are2 closely associated with direct manipulation
Fischer/Giaccardi/Ye/Sutcliffe/Mehandjiev
CACM-EUD
12
2.1 End-user Development
interfaces because they provide interactive objects such as windows and menus which
respond directly to mouse and keyboard interactions. However, direct manipulation
interfaces are not limited to GUIs. In the context of this thesis, AgentWeb provides
a direct manipulation interface for creating and managing game objects and drawing
them on the game scene.
Visual Programming Languages
Programming languages can be expressed by visual symbols and graphical notations
[23], [124]. Visual programming languages (VPLs) assist end users by presenting
a graphical representation of the programming constructs, which helps the user to
concentrate on domain-specific abstractions rather than syntactic ones, leading to an
increase in user productivity. Visual Programming Environments (VPEs) are closely
related to VPLs [20]: "When visual expressions are used in a programming environment as an editing shortcut to generate code that may or may not have a different
syntax from that used to edit in the code, the environment is called a visual programming environment (VPE)." Burnett and Baker have provided a classification of visual
programming languages [21].
Domain-Specific Languages
Domain-specific languages (DSLs) are known as small languages targeted for solving
certain problems in a specific application domain. Van Deursen and Visser [134] define
a domain-specific language (DSL) as "a programming language or executable specification language that offers, through appropriate notations and abstractions, expressive
power focused on, and usually restricted to, a particular problem domain." Fischer et
al. [41] provide guidelines on supporting domain experts in developing software artifacts, particularly with respect to the existing growth of the end user developers on
the Web.
Mernik et al. [79] observe that the development of a DSL rarely goes further than
developing an application library, since DSL development requires both domain knowledge and language development expertise. In many cases, a domain-specific embedded language (DSEL) approach is preferred, where the application domain libraries are
used together with a general-purpose programming language [55]. Developing a DSEL
reduces the the programming language development process of DSL to developing the
programming structures of the domain in a host general programming language.
The intersection of visual languages and domain-specific languages is domainspecific visual languages (DSVLs) [132]. DSVLs offer the benefits of visual languages,
e.g., freedom from syntax, and visual interactions, to define the behavior of a software
(object) using high level commands, constructs and abstractions.
Fischer et al. [40] discuss the evolutionary nature of domain modeling and construction that is done by a community of practice over time, and suggest supporting
13
2.1 End-user Development
collaboration in domain construction. Domain-Oriented Design Environments have
been suggested by Fischer et al. [36] to support human problem-domain interaction
rather than human computer interaction. End user programming environments such
as AgentSheets have been developed to enable domain experts to build their own DSLs
[110].
Interactive Programming
Interactive programming is the procedure of modifying a program while it is running,
and see the effect of changes as they are applied immediately. Interactive programming has been used in Smalltalk [50]. Interactive programming is a crucial feature for
a class of operating software whose execution can not be stopped. Often, interactive
programming languages are built on top of dynamic programming languages which allow the extension and modification of parts of the the running software. Such features
first appeared in the Lisp programming language.
Novices benefit from Interactive Programming Environments by watching the behavior of running software as they program it. They do not have to go through phases
of programming, then compilation/interpretation, then watching the behavior of the
software, then modifying it. Instead, they receive immediate feedback from the software as they modify it.
2.1.2 Collaboration practices among end users
Nardi [87] argues that end users tend to gain needed computer knowledge and solve
their computing problems by seeking help in their social environment, rather than
laboring in isolation. Such a pattern has been seen as a persistent activity and not the
exception in several computing areas including sharing UNIX customization files [70],
spreadsheets [88], and tailorable office systems [74], and CAD settings [46]. Such
modifications to a software is also known as “tailoring”. The collaborative tailoring
pattern has been demonstrated in other works as well [126] [75] [26].
Nardi [87] encounters a continuum of users with different skills ranging from end
users to local developers to programmers in a community of end users and highlights
the emergence of local developers in any settings where a computer program is widely
used. Local developers are defined as [87]: “domain experts who happen to have
an intrinsic interest in computers and have more advanced knowledge of a particular
program." Fischer et al. [38] extend this continuum to the consumer/designer spectrum ranging from passive consumer to well-informed consumer, end user, power user,
domain designer, and meta designer. Oberquelle proposes a classification of groupware tailoring based on collaborating actors and those who are affected by a tailoring
activity [95].
Maclean et al. [74] argue that tailoring has to become a community driven activity
based on the expertise of involved users. Mørch et al. [84] argue that the context in
14
2.1 End-user Development
which a software is being developed is different from the context in which that software is being used, and to reach a shared context, tailoring has to be done as the
indirect long-term collaboration between developers and users. They suggest “Multiple Representations” to capture and preserve a rich development context to be shared
among developers and users, and “Application Units” which make multiple representations accessible to end users at use time. Fischer et al. [38] introduce meta-design, a
conceptual framework aimed at providing social and technical infrastructures to support collaborative design, based on the assumption that future uses and problems can
not be anticipated at design time. They introduce the Seeding, Evolutionary Growth,
Reseeding (SER) process model for large evolving design artifacts. The SER model
proposes creating small pieces of artifacts as seeds by developers, which will be tailored by a number of end users during evolutionary growth step, where modifications
will be collected and adjusted to the system’s evolution during the reseeding phase.
Pipek et al. [95] categorize the existing collaborative tailoring works by time, distinguishing between three levels of intensity of user involvement with tool usage, namely:
“shared use”, “shared context”, and “shared tool” and introduce a fourth scenario called
“shared infrastructure”, and for each they discuss approaches to support collaborative
scenarios.
2.1.3 Collaborative and Social Software Engineering
Nowadays, software development is a collaborative process among the developers and
domain experts. Development activities involve a strong interaction between a set
of people that collaborate in order to achieve a common goal: produce an effective,
efficient software following the mantra “Maximum Results with Minimum Effort”. This
raises on the one hand the need for an effective way of sharing ideas and requirements
among the group members and on the other hand the need to create tools that can
support such collaboration. Together with my colleauges, we surveyed the literature
in the social aspects of software engineering [4].
Studies by Sawyer [117], DeMarco and Lister [31], and Jones [57] report a substantial amount of development process being spent in collaborative activities. Strubing [127] states three other activities that developers consistently carry out beyond
coding: (i) organizing their working space and process, (ii) representing and communicating design decisions and ideas, and (iii) communicating and negotiating with
various stakeholders. Booch has conducted an experiment to obtain a snapshot of the
daily life of a developer that conforms to Strubing’s findings [18].
The social aspects of the software process demand collaborative tools to support the
collaboration among team members. Particularly, software development is a highly distributed task, involving programmers, designers and architects who work in different
time/space, due to the growth of offshoring, outsourcing and open-source communities. The majority of large software projects are undertaken by teams of software staff
working across a number of organizations [90]. The computer-supported cooperative
15
2.2 Educational Game Design
work (CSCW) community has investigated computer-assisted collaboration for more
than two decades [51]. The CSCW application domains are numerous, but only those
collaborative tools built in order to assist in software development is of my rsearch
interest. Whitehead [143] distinguishes software engineering collaboration from the
collaboration in other domains as “artifact-based, or model-based collaboration, where
the focus is on the production of new models, the creation of shared meaning around
the models, and the elimination of errors and ambiguities within the models.” Sarma
[116] provides a classification framework for collaborative tools based on the user’s
effort required to collaborate effectively. At the highest layer of the framework, collaborative development environments (CDEs) provide a seamless environment where the
collaboration tools have become integrated into the development environment, thus
lowering the context switch barrier to the developers. Booch [18] defines a CDE as “a
virtual space where the stakeholders of a project, even if separated by time or space,
can meet, share, brainstorm, discuss, reason about, negotiate, record, and generally
labor together to carry out some task, most often to create some useful artifact and its
supporting objects.”
A number of CDEs have been proposed in the context of software engineering.
Boldyreff et al. [17] present two development environments to support collaborative
software engineering: GENESIS (GEneralised eNvironment for procEsS management
in cooperative Software Engineering) and OPHELIA (Open Platform and metHodologies for devELopment tools IntegrAtion in a distributed environment). Dossick et al.
[33] propose CHIME, a semi-automatically generated 3D virtual world representing
the software system. Users interact with project artifacts by “walking around” the virtual world, where they potentially encounter and collaborate with other users’ avatars.
CHIME aims to support large software development projects, in which team members
are often geographically and temporally dispersed, through novel use of virtual environment technology.
2.2 Educational Game Design
2.2.1 Game-Based Learning
Pong was the first video game, introduced in 1974. Since then, computer games have
come a long way, in terms of improving graphics and representing the real world.
Games are immersive and players spend enormous amount of time playing video
games. The effect of of digital games as a learning tool has been recognized [97]
[48] [68] [76]. Prensky states [96]: "... important message that all parents and educators need to hear: Video games are not the enemy, but the best opportunity we have
to engage our kids in real learning." Wu and Richards state [148]:“Well-designed digital games can facilitate learning because they are designed to contextualize learning
based on a set of learning principles. Principles such as immediate feedback, sandbox,
16
2.2 Educational Game Design
customization, and adjustable difficulty motivate players to work within their regime
of competence with the problem space presented by the game [47]."
Serious Games and Gamification
In the 21st century, using games beyond entertainment has gained a lot of attention,
which has led to the rise of two concerns: Serious Games and Gamification. The Serious
Games Initiative was started in 2002 by Ben Sawyer and David Rejecsk which brought
together academia, industry, and military to build games for training and simulation
purposes [81]. Gamification is the use of game elements and game design techniques
in non-game contexts such as economics and business [32] [140]. Serious Games are
different from Gamification. Serious Games are digital games and simulations that
have been built for teaching and training purposes, whereas Gamification takes advantages of game design techniques and elements to address the problems in other
fields. These ideas have been pushed further to design games that address global problems such as climate change and health [77]. Fun and immersion are characteristics of
games that are used to address threats such as boredom, anxiety and distraction, cognitive states of mind that are also associated with the theory of flow[28], in the activities
that require concentration and engagement. Educational games and simulations are
being increasingly adopted in public schools [54].
Generation Y
The improvements in several computing fields such as AI and computer graphics, coupled with the rise and ubiquity of computing power manifested by the growth of smart
and mobile computing devices are triggering but are not the only reasons for video
games to take off. Such growth is partly due to the new generation of learners known
as Generation Y, Millenials, Net Generation, or Gamer Generation, those who were
born roughly between 1982 and 1997 [129].
A 2005 national survey of 8-18 year olds in the United States revealed that each
week young people spend as much as a full-time job on media [114]. One of the profound characteristics of the Net generation is multitasking, i.e., they often use several
forms of media such as listening to music, doing homework, and following a TV show
conversation on the side, all at the same time. They are constantly communication
with friends through mobile phones, SMS, chat, email and social media. These characteristics suggest the use of games in education to accommodate the Net Generation
[13].
2.2.2 Educational Game Design Environments
In the first chapter I defined the term ‘educational game design’ in the context of this
thesis. In this section, I discuss some of these environments as example and discuss
17
2.2 Educational Game Design
how they support novices in programming activities.
Educational programming languages and environments have been created since
Logo in 1967. Kelleher and Pausch [61] provide a comprehensive survey on these languages, their educational purposes, their programming paradigm, and so on. Since the
early 90s, educational programming environments started to form education around
game design activities. Accordingly, the learner is involved in designing and programming graphical objects that interact with each other in a game scene.
AgentSheets
Figure 2.3. AgentSheets Game Design Environment
AgentSheets1 , created by Alexander Repenning, appeared as early as 1991, introducing agent-based graphical rewrite rules [101]. AgentSheets was further enhanced
by introducing a drag-and-drop Tactile Programming language called Visual AgenTalk
in 1996 [104], [106] (Figure 2.3).
AgentSheets provides users with a visual IDE for designing and programming games
and scientific simulations. The game design model employs the spreadsheet structure
to define the location of game objects. Accordingly, the game scene is a 3D grid of
game objects which interact with their adjacent objects. Game objects, called Agents,
are programmed by the novice programmer. Each agent consists of one or more depictions as its visual representation in the game scene and a program. One drawback in
AgentSheets is that the motion of agents between the cells are not animated.
1
http://www.agentsheets.com
18
2.2 Educational Game Design
The programming of agents is assisted by a visual programming environment which
allows the user to compose programming instructions using drag-and-drop. The programming paradigm used in AgentShees is rule-based. Each rule contains a set of conditions and actions. Conditions and actions form a domain-specific language to define
the possible behavior of agents during the game. In the recent version of AgentSheets,
conversational programming has been added to help learners receive immediate feedback about the behavior of the program at run time [102].
AgentSheets has been widely used in public schools [111]. In particular, the Scalable Game Design project [103] researches and implements a game design curriculum
at public schools with a focus on theory of flow [28], computational thinking, and
transfer [11]. Field experience with the use of AgentSheets in schools and universities
show increasing engagement and interest in the field of computer science [12].
KidSim / Cocoa / Stagecast Creator
Figure 2.4. Stagecast Creator Game Design Environment
KidSim2 [30, 125], originally created by Allen Cypher and David Canfield Smith,
was shaped around 1994 , which was then renamed to Cocoa and later on to StageCast
Creator (Figure 2.4).
Similar to AgentSheets, Creator offers a grid-like game scene for positioning the
2
http://www.stagecast.com
19
2.2 Educational Game Design
objects. However, the movement of objects between the grid cells is animated. Creator employs rules to define the behavior of objects in a visual programming environment. Creator uses programming by demonstration to lower the programming barrier
to novices. Accordingly, the user demonstrates the expected behavior of an object to
the system rather than programming it [29].
Squeak Etoys
Figure 2.5. Squeak Etoys Game Design Environment
Squeak Etoys [59] was designed by Alan Kay and appeared in 1997. Etoys is
a media-rich environment for creating objects and programming them. The programming is supported by graphical tiles similar to those in AgentSheets, which the user can
drag-and-drop to compose the behavior of the objects (Figure 2.5). Etoys is shipped
on more than one million OLPC laptops [43].
Alice
Alice is a programmable authoring tool to create interactive 3D worlds [27]. Similar
to earlier environments, Alice comes with a visual programming environment to create
computer animations using 3D models, and supports an object-oriented, event driven
20
2.2 Educational Game Design
Figure 2.6. Alice 3D Game Design Environment
model of programming (Figure 2.6). A storytelling version of alice was created by
Kelleher and shown to increase motivation among young girls [62].
Game Maker
Game Maker was initially developed by Mark Overmars [93]. The original version was
used to create 2D games. Most recently it has added support for creating 3D games.
The Game Maker UI makes heavy use of parameterizing the game objects through
forms and dialogs. It consists of a scripting language that is affected by C and Java to
define the behavior of objects. Moreover, it adds a drag-and-drop visual interface on
top of the scripting language (Figure 2.7).
Scratch
Scratch is designed by Mitchel Resnick [112]. It is a 2D media-rich environment targeting children and computer clubhouses. It provides a visual programming environment
containing programming blocks that can be composed using drag-and-drop. Composing the blocks is further supported with a novel representation that allows snapping
21
2.2 Educational Game Design
Figure 2.7. Game Maker Game Design Environment
the blocks together if they are syntactically meaningful (Figure 2.8). Scratch has a very
active community of users who have shared more than two million creations through
its Website.
Greenfoot
Greenfoot is an interactive Java development environment designed primarily for educational purposes at the high school and undergraduate level [53]. It allows easy
development of two-dimensional graphical applications, such as simulations and interactive games. Greenfoot provides novices with an interactive visual world, and an
integrated programming environment for writing Java code with syntax highlighting,
code completion, and project management (Figure 2.9).
22
2.2 Educational Game Design
Figure 2.8. Scratch Game Design Environment
Kodu
Kodu is a proprietary game design environment created by Microsoft’s FUSE Lab [72].
The user programs the behavior of objects in a 3D world using a rule-based programming language resembles those used in AgentSheets, Etoys, and Alice [73] (Figure
2.10).
AgentCubes
AgentCubes is the 3D successor of AgentSheets [108]. It has been designed by Alexander Repenning. It inherits most of the programming blocks from AgentSheets, while
extending their capabilities to be used in the 3D world. The 3D world design is also
supported with direct-manipulation visual tools (Figure 2.11).
Non-educational Game Design Tools and Languages
There are several other game design languages and environments that are used for
empowering developers to develop games rapidly. These environments are often proprietary software and are not used for educational purposes. Here is a partial list of
these languages and environments:
• Lua is a scripting language that is widely used in game developer communities3 .
3
http://www.lua.org/
23
2.2 Educational Game Design
Figure 2.9. Greenfoot Game Design Environment
With its easy-to-use C API, Lua is intended to be embedded into other applications.
• Microsoft XNA is a set of tools and a runtime environment to facilitate the development of video games4 .
• GameSalad Creator is a visual authoring tool with drag-and-drop programming
5
.
• Corona6 is a software development kit (SDK) for programming video games. It
integrates with Lua and is used often for creating games for mobile platforms.
• Stencyl7 is a game development platform creating 2D games for mobile devices
and the Web.
4
http://msdn.microsoft.com/xna
http://gamesalad.com/
6
http://www.coronalabs.com/
7
http://www.stencyl.com/
5
24
2.2 Educational Game Design
Figure 2.10. Kodu Game Design Environment
2.2.3 State-of-the-art of support for collaboration in GDEs
Only a few of the end-user programming languages provide support for collaboration.
Kelleher and Pausch [61] identify three types of collaboration that are supported in
different programming languages. (i) side-by-side-based collaboration in which two or
more users were manipulating the same program on computers that were located in
the same room, (ii) networked-shared manipulation in which users were in different
locations but connected to a common network and could collaborate while building a
program, and (iii) networked-shared results in which users were in different location
but connected to a common network and could share completed programs or program fragments. Based on their survey of fifty eight end-user programming languages,
there are only ten models supporting one of these three types, among six of which
support side-by-side collaboration, only one supports networked-shared manipulation,
and three support networked-shared results. In none of the developed EUP models can
end users collaborate from inside the programming environment at the time of writing
the program.
There have been some efforts to enable sharing and collaboration among game
designers. Early on, Behavior Exchange let users share AgentSheets games through
the Web [107]. The Game Maker community shares its games through a Web site8 ,
although online playing or exploring of the games is not possible. The Scratch Web
site has collected millions of user-created games9 .
Users upload their creations to a supplementary Website in order to share them
8
9
www.yoyogames.com
scratch.mit.edu
25
2.3 Computational Thinking
Figure 2.11. AgentCubes Game Design Environment
with others. The others can play the game from inside the browser, but they need to
download the games to their machine in order to browse or modify the inside of the
game, including its graphic design and program. In case the user modifies the game
and would like to share it again with others, she needs to upload the game back on the
Website. On the Website, learning is limited to discussion forums and comments on
the game, rather than sharing and collaboration on the game design process. I discuss
the limitations of existing models for sharing games through Websites in Section 5.1.
2.3 Computational Thinking
In an emerging view on the general benefits of computing literacy, whether the students eventually pursue computer science or any other discipline, the term ‘computational thinking’ has gained much attention [147]. Computational thinking is the
use of problem solving techniques that rely on the fundamental concepts of computer
science.The wider impact of gaining computational thinking skills, in addition to just
learning how to program, have only recently been recognized.
Although game design environments have focused on teaching computer programming to novices, recently they have served as a platform to learn and practice computational thinking skills only implicitly. Game Design is considered to support gaining
computational thinking skills [148]. Recently, game design courses with a focus on
computational thinking have been designed, taught and reported [60] [120]. In these
26
2.3 Computational Thinking
courses, educational game design environments are used to help students gain computational thinking skills [111].
A closely related topic to computational thinking is ‘software design thinking’ which
brings up the kind of thinking that is required in software design, such as abstraction,
composition, and relationship between objects, rather than algorithmic design [52].
2.3.1 Computational Thinking Patterns
Computational thinking patterns (CTPs) are algorithms that occur repeatedly in a computational domain, e.g., video game design [56]. As an analogy, CTPs are abstractions
similar to design patterns [45], that model a solution for a recurring software design
problem. In game design environments, such patterns occur across different games
such as: moving the game character with keyboard, or detecting the collision between
the game objects.
In AgentSheets, computational thinking patterns are recognized and used as a core
concept for preparing educational materials. The game design model presented in this
thesis follows the same game design model of AgentSheets and therefore has similar
computational thinking patterns. Here I cite the description of those patterns which
are referred to in this thesis from Scalable Game Design Wiki10 :
• Cursor Control: This is the most basic pattern in game design. The users can
control/move their agents with arrow keys or any other keys.
• Collision: Collisions deal with the event wherein two agents run into each other.
An example of the collision pattern occurs in the gamelet Frogger, where it can
happen that a moving truck hits a frog. The truck wants to continue driving to
the right but the frog is in the way. The live frog needs to be replaced by a dead
frog when this happens.
• Generation: In many gamelet situations one agent needs to generate a stream
of other agents that move away, for example vehicles coming out of a tunnel,
or electrons coming from an electron gun. The generator agent should create
a generated agent next to itself (say to the right), with some probability. The
generated agents, once created, move away to the right.
• Absorption: This is the opposite of the generate pattern. Here something needs
to absorb a stream of agents, for example a tunnel mouth absorbing vehicles,
or a target absorbing electrons. The recommended approach is that the to-beabsorbed agents look for the absorber just ahead as they move. If the detect it,
they erase themselves.
10
A list of all computational thinking patterns of AgentSheets and their description can be found in
here: http://scalablegamedesign.cs.colorado.edu/wiki
27
2.4 World-Wide Web
• Push: Another common behavior is pushing. In the game of Sokoban a person is
supposed to push boxes around a maze onto targets. The box should only move
if it could move onto a target spot or a regular ground tile. If the box is blocked
because it is facing a wall in the direction of the push then neither it nor the
person pushing it should move.
More advanced computational thinking patterns include Pull, Transport, Diffusion, Collaborative Diffusion, Hill Climbing, Multiple Needs, Perceive-Act Synchronization, Poll,
Script, and Seek. These patterns are not discussed in this thesis.
2.4 World-Wide Web
Tim Berners-Lee put together the Internet and hypertext to build a global information
medium [16]. Berners-Lee and others originally developed HyperText Transfer Protocol (HTTP), HTML, Web server, and a Web browser with which users read and write
documents over the Internet. Each document is globally identified through a uniform
resource identifier (URI) and is accessible via a Web browser. The Web has brought us
an information universe which has changed how people live and work [15].
The Web has had a great impact on software engineering. Oreizy et al. [92]
consider the Web as an enabling technology with the potential to change software development by providing a cheap medium for large-scale software distribution, update,
and reuse. Open hypermedia systems [80] had been well investigated even before
the invention of the World Wide Web, and had been used in the context of software
engineering [10].
HTTP and its support for Representational State Transfer (REST) [35] architectural
style for engineering of Web-based software has brought scalability and simplicity for
creating Web-based applications. Being available world-wide and highly scalable, the
Web is one of dominant platforms for collaboration. Booch and Brown [18] provide a
survey of Collaborative Development Environments. According to them, such collaborative sites, mainly built on top of the Web, can be classified into six groups based
on their application domain: non-software domains, asset management, information
services, infrastructure, community, and software development. They suggest a list of
features for CDEs to be added to the feature list of virtual meeting spaces provided by
Fournier [42]. They organize the CDE features into three categories, namely: coordination, collaboration, and community building. Furthermore, they propose a conceptual model for a software development-specific CDE, consisting of three layers: project
workspaces, team tools and development resources.
2.4.1 Participative Web
Although Berners-Lee’s original vision of the Web was a read-write medium, the Web
continued to grow as a read-only medium. The first efforts towards building a read-
28
2.4 World-Wide Web
write Web by Whitehead and others resulted in developing WebDAV which extends
HTTP [142] as an infrastructure for asynchronous collaborative authoring on the Web.
With the improvement of Web technologies and specifically Asynchronous JavaScript
and XML (AJAX), Web-based user interfaces greatly improved and opened the way
for the Web to go beyond its initial use for sharing the documents. Having such infrastructures in place, the Web has evolved from an information platform to a social
platform for sharing, collaboration, participation, and education during the last decade
[136]. Web-based communities such as Wikipedia, YouTube, Flickr and the growth of
Web-based social networking applications such as Twitter and Facebook popularized
the term ‘Web 2.0’ as a platform on which the focus has shifted from reading and
writing static Web documents to sharing and participation in a Web-based community.
As a result of such participative culture [37], a Web-based platform, e.g., Wikipedia,
becomes a global entity which could not have been built by any one person or organization. Such phenomena are known as ‘Collective Intelligence’ [91]. The Web
has been repeatedly used to build such communities in several domains. Wikipedia,
YouTube, Flickr, Stack Overflow, Twitter, Cragislist, IMDB, eHow, and Quora are just a
few instances to name.
Participative Web applications have been used in the software engineering process.
Wiki systems are used for knowledge management during different software engineering phases. Using wiki as a collaborative development environment is one step beyond
using it only as a documentation repository [67]. Xiao et al. [149] present Galaxy
Wiki, a wiki based environment in which programmers can collaboratively write source
code, build projects, execute programs and debug defects. Cheyer and Levy [24] introduce WubHub, a wiki-based collaborative programming environment with the notion
of executable pages that act as functions, i.e., are executable and able to call each
other and pass defined data types. Users can collaboratively write and compose services, and build upon each others’ work, assembling new services that provide new or
more customized features. Open-source communities such as Github11 , Google Code12 ,
CodePlex13 , BitBucket14 and SourceForge15 have leveraged the Web as a community
building platform around open-source projects [99].
2.4.2 Web 2.0 for Education
Web 2.0 has been recognized as a medium that supports social learning. Reed et al.
[100] argue that:“ to be considered social learning, a process must: (1) demonstrate
that a change in understanding has taken place in the individuals involved; (2) demonstrate that this change goes beyond the individual and becomes situated within wider
11
https://github.com/
http://code.google.com/
13
http://www.codeplex.com/
14
https://bitbucket.org/
15
http://sourceforge.net/
12
29
2.5 Web as a runtime platform
social units or communities of practice; and (3) occur through social interactions and
processes between actors within a social network."
According to Wenger [66] [139], a community of practice is characterized by “a
shared domain of interest" where “members interact and learn together" and “develop
a shared repertoire of resources." Downes [34] mentions: “In the world of e-learning,
the closest thing to a social network is a community of practice," and explains the shift
from traditional learning management systems (LMS) in which teachers publish the
learning content and students read them, to a platform in which students develop and
share their knowledge throughout their learning process.
McLoughlin explains the opportunities and constraints of Web 2.0 and social software for pedagogy, and defines social software as “an open architecture that facilitates user-controlled, collaboratively generated knowledge and community-focused
enquiry”, which provides a suite of tools to support greater learner choice and selfdirection [78]. Ravenscroft [98] discusses the use and implications of social software
for learning, including the need to tailor the technology for learning purposes. Alexander [8] brings together examples of social software and how they can be used in education. Vassileva [135] explains how social learning technologies can be designed using
existing technologies: ontologies versus social tagging, exploratory search, collaborative versus self-managed social recommendations, trust and reputation mechanisms,
mechanism design, and social visualization, in order to support the learners to find the
right learning material and the right people on demand.
2.5 Web as a runtime platform
2.5.1 Programmable Web and Mashups
With the advent of AJAX, professional programmers mixed the online data and services
as Web Mashups16 . Several mashup tools such as Yahoo! Pipes 17 have been developed
to simplify the mashup development tasks by combining visual and data-flow programming techniques. End user programming approaches such as data-flow programming,
scripting, and spreadsheets have been employed to lower the mashup development
barrier for novice programmers. For instance, Mashroom supports end-user mashup
programming using nested tables and spreadsheet-like programming [138].
Although mashups are similar to games in extending the static boundaries of Web
applications by enabling professionals and end-users to develop executable programs
and run them in the browser, they are fundamentally different because the response
time is often not a strict constraint, at least in end-user mashup development environments. Mashup environments often take advantage of AJAX to run the business logic
on the server and the browser just provides a user input interface. AgentWeb has high
16
17
http://www.programmableweb.com/
http://pipes.yahoo.com
30
2.5 Web as a runtime platform
interactivity demands that distinguish it from mashups and enforces a full client-side
architecture to avoid network latencies that can be imposed by AJAX.
2.5.2 RIA Platforms vs. Open Web for Games
To date, applications in many domains have been ported to the Web. In the domain
of games and interactive applications, though, the lack of native support for developing interactive games with advanced graphics led to the development of Web-based
games using Java applets or proprietary ‘Rich Internet Application’ (RIA) platforms
such as Adobe Flash [131]. Such games are developed by (professional) programmers and delivered to users by embedding them into Web pages. There have been
efforts to leverage RIAs to enable end users to create their own interactive applications. Websites such as mamamedia.com, kerpoof.com, gamestarmechanic.com, and
mygamebuilder.com represent a range of such tools. While these tools are more focused on creating artwork and storytelling, the extent of programming in them is limited to modifying a static set of parameters to define the behavior of game characters
rather than composing their behavior as in AgentWeb. Programming by composition
will allow a higher variety of games to be built in AgentWeb. Some of the existing
game design environments such as AgentSheets and Scratch are capable of exporting
their games as Java applets so that the game can be played online. However, the game
design remains a solitary desktop-based experience to their users.
Although most games are still delivered to the Web using RIAs, several casual
games have been developed using open-Web technologies, even before the HTML5 era.
Examples of these games include Lemmings18 and Google Pac-Man19 . The emergence
of HTML5 and the increasing speed of JavaScript engines have led to the development
of open-Web games20 and game engines21 . Environments such as Play My Code 22
enable the HTML5 game programming right inside the browser. However, all these
interactive games are developed by (professional) programmers. Play My Code, for instance, lets programmers develop their games using an object-oriented programming
language called Quby and the game design involves low level implementations of game
objects and graphics rendering. Prior to AgentWeb, developing open-Web games had
remained out of reach of novice programmers.
18
http://www.elizium.nu/scripts/lemmings/
http://www.google.com/pacman/
20
Such as those contested in Mozilla’s Game On event:
https://gaming.mozillalabs.com/
21
An extensive list of HTML5 game engines is provided in:
https://github.com/bebraw/jswiki/wiki/Game-Engines
22
http://www.playmycode.com/
19
31
2.6 Conclusions
2.6 Conclusions
In this chapter, I described the related works that define the scope and foundation for
this research. End-user development and educational game design have a long history,
while computational thinking and participative Web are relatively new. From the historical viewpoint, this thesis applies the newly developed concepts and technologies of
participative Web and computational thinking to the long-practiced educational game
design. In the next chapter, I describe how the new technologies, in particular, openWeb technologies, are employed to enable the game design practices in a cyberlearning
platform.
32
2.6 Conclusions
Chapter 3
Building an Open-Web Game Design
Environment
I employed open-Web technologies, i.e., HTML5, JavaScript, and CSS, to build an educational game design environment. The environment, called AgentWeb, lets users
practice the game design and programming activities right inside their browser, i.e.,
no need for desktop-based clients or third-party proprietary software such as Flash
to run the application. Such an environment has obvious attractions as discussed in
Chapter 1: availability across platforms, instant access by all Web users, enabling cooperative game design following the Web 2.0 community-building patterns, and potential
for integration with smartphone platforms and the users’ social network to seamlessly
integrate into the users’ platform of choice. The Website around the game design environment enables game designers to cooperation around designing the game artifacts.
The support for cooperation is discussed in the next two chapters. This chapter focuses
on the design and implementation of the open-Web game design environment as the
core component of a collaborative game design platform. Parts of this chapter were
published in [5].
In the rest of this chapter, in Section 3.1 I describe the requirements for creating
AgentWeb. In Section 3.2 I explain the challenges of creating an educational game
design environment using open-Web technologies. Section 3.3 discusses early feasibility experiments through which I developed two main components of AgentWeb:
an HTML5-based a game engine and visual programming environment. Section 3.4
presents the initial design of AgentWeb’s user interface. In Section 3.5 I present the
details of designing and implementing AgentWeb to meet its requirements. Section 3.6
describes the details of the visual programming language. 3.7 describes the runtime
system of AgentWeb. Section 3.8 reports on the performance issues that I faced during
the development of AgentWeb and how I overcame them. Section 3.9 provides the
performance evaluation results across a set of platforms. Section 3.10 provides the
results of an initial usability study of AgentWeb. Section 3.11 concludes the chapter.
33
34
3.1 System Requirements
3.1 System Requirements
In this section I discuss the requirements for AgentWeb including choosing a game
design model suitable for novice programmers and choosing the platform to encourage
sharing and collaboration. Afterwards, we discuss the extent of using the Web as
a programming platform and reasons I chose to develop AgentWeb using open-Web
technologies.
3.1.1 Choice of game design model and support for end users
Rather than designing a game design model from scratch, I chose to reuse an existing game design model that has been successful in addressing the needs of novice
programmers and has been widely used in educational environments. In particular, I
chose the game design model of AgentWeb to resemble that of AgentSheets, which has
been successful in teaching computational thinking through game design worldwide,
and specifically in game design courses in schools [103]. AgentSheets combines the
benefits of visual programming and spreadsheets to simplify creating 2D games. It
consists of a visual rule-based domain-oriented programming language called ‘Visual
AgenTalk’ [105]. I described AgentSheets in more details in Section 2.2.2.
Following the AgentSheets game design model, I required that the game design
environment supports novices using a visual IDE including support for creating game
objects, programming them and drawing the game scene, all through a visual interface.
The environment should be capable of running the games created in the high-level
language in a reasonable speed required by interactive games. Moreover, I added two
requirements for the environment to further support novices:
• Direct manipulation lets users modify game objects through a visual interface and
the results are immediately applied across the environment [122]. For instance,
if the user updates the depiction of a game object, the update should be instantly
available in the game scene. This requires a response time of 100ms or less [89].
• Interactive programming lets users modify the program of a game and the changes
in the program are instantly applied in the game, even while the game is running. A typical game design use-case of interactive programming is in tuning the
speed of moving objects, e.g. cars, in the game. In the absence of interactive
support, a game designer would have to recompile the program manually after a
modification and re-run the game to explore if the object behavior is satisfactory.
3.1.2 Choice of collaboration platform
Existing game design environments lack support for collaboration among novices.
AgentWeb was designed to address this issue by creating a collaborative environment
35
3.1 System Requirements
around the game design activities. I envisioned the following requirements to enable
users to achieve the most out of AgentWeb with the least effort:
• Platform independence is the fundamental requirement for enabling users to integrate an application into their lifestyle. Users should be able to access AgentWeb
regardless of the device and operating system they use.
• Instant exploration of the inside (design) of the game is needed to ease sharing
and learning the design of the game, and not only the play of the game. Existing game design environments enable sharing the games through a supplementary Website but exploring the design of a game, as a way of learning, requires
downloading the game to the desktop and to open it inside the environment.
AgentWeb should transcend this barrier by allowing for exploring the design and
programming of a game created by others with no extra effort. This matter is
discussed in details in Section 5.1.
• Community building features should be provided to users to let them gather and
collaborate around a game. Accordingly, a game becomes a software artifact
that lets users collaborate in a many-to-many fashion, e.g., similar to Wikipedia,
while building it. This is in contrast to existing game sharing models in which a
single user creates the game and shares it with others in a one-to-many fashion.
Furthermore, AgentWeb should integrate into the user’s online social networking
presence, such as Facebook, to ease the community building process with people
whom the user already knows.
To realize these requirements, I considered three platforms as options to build
AgentWeb:
• A desktop application enhanced with networking and cloud capabilities to enable
sharing and collaboration among users;
• A mobile application for smart devices that employs intrinsic networking capabilities of such devices to connect to a cloud platform that enables sharing and
collaboration among users;
• A Web application directly accessed from the Web browser. The Web-based application follows Web 2.0 community-building best practices to enable sharing
and collaboration among users.
I chose the third option, the Web, as the platform because: (i) The Web is a crossplatform environment that can be accessed from both desktop and mobile platforms
regardless of their operating system and hardware. On mobile devices, though, applications will require tailored interaction design to make them usable. (ii) As previously
discussed, participative Web has been highly successful in building communities and
enabling social learning in numerous applications from general social networking to
highly-specific communities (See Section 2.4.1).
36
3.2 Challenges
3.1.3 Choice of Web technology
Games are interactive applications consisting of multimedia such as graphics, sound,
and user input. As mentioned earlier, the lack of support for delivering multimedia
content in the browsers initially led to developing games using Java and Flash and
embedding them in Web pages.
Recently, HTML 5 has included advanced support for multimedia content including
video, audio and graphics. Simultaneously, with the growth of the Web as an application development platform, the JavaScript engines have improved drastically. This has
enabled the opportunities of developing open-Web games using only HTML5, CSS, and
JavaScript. I chose open-Web technologies to build AgentWeb because:
• Open-Web technologies are non-proprietary standards supported by the majority
of modern Web browsers;
• The browser does not require any third-party plugin to run the game design
environment or the games built using the environment;
• They are highly cross-platform, in particular on mobile devices, compared to
Flash.
3.2 Challenges
As software, an educational game design environment consists of components that go
beyond the static HTML pages and make it challenging to build using open-Web technologies. Such components include a visual domain-specific programming language
and its compiler, a graphical scene authoring tool, a game engine and its graphic rendering engine, all combined in an integrated development environment (IDE) suitable
for novice programmers. Developing such a software poses design and implementation
challenges such as performance and usability.
First, HTML is originally designed to represent static documents, i.e., a Web page
is an HTML document located on the server, it is loaded and rendered by the client’s
browser. The interaction with the Web page was limited to navigation through links
and sending data and files to a server through a form. Only the drastic success of the
Web and need for full-fledged Web applications led to the creation and standardization
of the Document Object Model (DOM) and Cascaded StyleSheets (CSS) which allow
dynamic modification of a Web page after the page is loaded in the browser. With the
advent of XMLHTTPRequest API1 , AJAX-based applications could dynamically load any
piece of HTML or data into the same page, without navigating to another URL, which
allowed creating Web applications rather than static Web pages. However, support for
1
http://www.w3.org/TR/XMLHttpRequest/
37
3.3 Feasibility Studies
representing multimedia such as audio, video and advanced graphics, which are the
essential components of a game, were missing until the advent of HTML52 .
Second, JavaScript was initially developed as a light-weight interpreter-based scripting language to handle simple user interactions with the Web-page. By no means could
Web technologies at their early stages be used, even by professional developers, to develop and execute interactive applications such as games which demand high graphics
and execution performance. During the last few years the performance of JavaScript
engines has greatly improved using just-in-time (JIT) compilation techniques, such as
those used in SpiderMonkey [44], and full-compilation as used in Google V83 , which
has led to creating several open-Web games and game engines4 .
Third, even though the Web technologies have been improved in the last few years,
game development on the Web has been restricted to the professional programmers.
Enabling game design for novices, in particular for educational purposes, requires providing EUD tools, such as a visual drag-and-drop programming language, as discussed
earlier. Building such rich user interfaces using low-level DOM elements and events
is more challenging compared to other programming platforms, because even simple
interactions such as drag-and-drop are not provided by the platform and therefore
need to be simulated. Furthermore, such high levels of abstractions impose further
performance overhead on the execution of the game.
Table 3.1 summarizes the challenges of building such an environment using openWeb technologies, together with my suggested solution, corresponding evaluation measure and success metric for each challenge. The suggested solutions in this chapter
are applicable to similar open-Web software that involve building highly responsive
IDEs beyond current AJAX-based applications, designing and executing high-level visual programming languages, or executing highly interactive graphical applications.
3.3 Feasibility Studies
3.3.1 Ristretto Mobile: an AgentSheets to JavaScript Compiler
The first step towards building AgentWeb was to explore the feasibility of running
interactive games inside the browsers. This would let me decide whether the Web is a
hospitable platform for running interactive games, i.e., I explore how to overcome the
first challenge mentioned in Table 3.1.
To push the limits of using Web technologies to the extreme, I set my aims high,
to render the games inside the browser of mobile devices. This would add further
challenges, mainly due to (i) the low resources of mobile devices including computing
2
http://dev.w3.org/html5/spec/Overview.html
http://code.google.com/p/v8/
4
A
large
list
of
existing
JavaScript-based
https://github.com/bebraw/jswiki/wiki/Game-Engines
3
game
engines
can
be
found
in
38
3.3 Feasibility Studies
Challenge
Solution
Evaluation
Measure
Success Metric
The dynamic and
interactive nature of
games is pushing the
capabilities of HTML
which traditionally
focused on static
representations.
Employ JavaScript
to build highly
interactive
Web-native
applications such
as games.
Usability and
playability of
games
Should be able to
run 2D interactive
games in the Web
browser.
End-User
programming: enable
Web users with no
programming
background to create a
game from scratch.
Build a high-level
programming tool
in JavaScript
based on
established
principles of visual
end-user
programming.
Empirical
user study
End users such as
high school
students with no
programming
background can
use tool to make a
complete game.
The performance
required to run games
including simulations
and visualizations may
be beyond the
computational
capabilities of
JavaScript.
Build performance
optimized
implementation of
runtime and
visualization
mechanisms based
on profiling.
Performance
evaluation for
characterizing the
performance
of interactive
applications
JavaScript
generated by tool
should yield
performance
required by
interactive
applications.
Table 3.1. Challenges of creating an educational game design environment using
open-Web technologies and corresponding proposed solutions.
39
3.3 Feasibility Studies
power and memory; (ii) high variety of used hardware among mobile devices. Accordingly, if I could reach a point to run a game in different mobile devices, it would
guarantee the universality of the technology I was relying on to the maximum possible,
i.e., platform independence, as mentioned as a requirement of choice of collaboration
platform in Section 3.1.2.
One major decision was the choice of graphics rendering technology. When I
started AgentWeb, HTML5 did not exist yet. The possibilities to render the graphics in the browser natively, i.e., without any plugin, are Canvas, SVG, or raw HTML.
The development was done in pre- iPad and Android era. First-generation iPhone was
just launched in January 2007 which provided one of the very first mobile surfing experiences with full JavaScript support using mobile Safari. Nokia produced small tablets
called Nokia N800 which had a working Web browser based on FireFox. At the time,
SVG was not supported for mobile devices. Canvas API was originally developed by
Apple Inc., and was offered in WebKit which is the rendering engine behind the Safari
browser. Mozilla FireFox also supported Canvas API. I decided to use Canvas as it provides features over using raw HTML such as pixel-level access to the game scene, and
possible hardware acceleration by graphics cards (which was introduced later). Back
in 2007 I developed the first graphics engine of AgentWeb using Canvas. To maintain the open-ness of AgentWeb, just in case Canvas would not have been supported by
other browsers, I also developed and maintained a HTML-based graphics engine which
was rendering the game objects using HTML table and CSS absolute positioning. Later
on, Canvas found its way into HTML5 specification to become a W3C standard to be
supported in all the browsers. I had a good choice of technology!
In late 2007, I developed a compiler which turned an AgentSheets game into a pure
JavaScript executable running in the browser, called Ristretto Mobile, or in short RM 5 .
Figure 3.1 shows the architecture of Ristretto Mobile. RM has three main components:
(i) an application retriever which downloads an AgentSheets application, including
agent depiction files (in GIF format) and agent behavior (program) files (in XML format) from the server to the client’s browser; (ii) A compiler, written in JavaScript,
which right inside the client translates the program of each agent to an equivalent
JavaScript program; (iii) a runtime system, mainly composed of a game engine and a
graphics engine, initializes the game scene with the instances of agent objects, executes
their program, and renders the graphics on a Canvas.
I succeeded in compiling and running 14 AgentSheets examples, on both firstgeneration iPhone and Nokia N800 devices. Figure 3.2 depicts an iPhone running a
sokoban game in Mobile Safari compiled by Ristretto Mobile. The execution speed was
fairly low, often below 10 frames per second (fps), due to the low speed of mobile
browsers and low computing power of mobile devices. Nevertheless, the games were
fully functional. We could reach more than 30fps, the threshold for running a fastpace game, on a typical personal computer of the time, e.g., a 2.16 GHz Intel dual-core
5
Ristretto Mobile is available at http://www.inf.usi.ch/phd/ahmadi/rm/index.html
40
3.3 Feasibility Studies
Web Server / File System
AgentSheets Application Files
AJAX-based retriever downloads
project files to the client's browser
Behavior (Program) Files
HTML5 Compiler translates the
program files to JavaScript objects
Program Files translated to JavaScript
HTML5 Runtime (including a Game
Engine and a Graphics Engine)
executes compiled behavior objects
HTML5 executable
Client' Web Browser
Client
Figure 3.1. Ristretto Mobile compiles AgentSheets applications to HTML5 to be
executed inside the Web browser.
processor. With this, I validated the potential of using desktop-based browsers as a
hospitable platform for running end-user developed games. The full work is reported
in [7].
3.3.2 Web-based Visual Programming of Physical Devices
After succeeding in executing interactive games inside the browser, I started exploring
how to build a high-level visual programming language inside the browser. Not only
did I want the visual interface of the language to be built using open-Web technologies, but I also required to execute the language at the client side, by either interpretation or compilation of the visual program to JavaScript. To put the development
challenges into the research-oriented perspective, and in a team effort, together with
my colleauges we explored the idea of controlling physical devices through the Web
by means of defining the behavior of the device using a visual programming environ-
41
3.3 Feasibility Studies
Figure 3.2. A first-generation iPhone running a Sokoban game inside Mobile Safari.
The game was created by AgentSheets and compiled to HTML5 by Ristretto Mobile.
ment. For instance, the user could use a Web page to compose the functionality of a
Lego Mindstorms NXT6 , a programmable robot, using a visual rule-based language.
I was responsible for designing and developing the Web-based visual programming
environment. As the programming paradigm, I chose a visual rule-based programming
language similar to ‘Visual AgenTalk’, the programming language of AgentSheets. The
language consists of conditions and actions which are uses to compose rules. Rules are
tested continuously during execution and when the conditions of a rule are satisfied,
its actions are executed. Figure 3.3 shows the user interface of the visual programming environment7 . I used an interpreter to execute the rules that were composed by
the user. Therefore, no code translation was done, which increased the development
speed.
An important aspect of the work was that as each device has its own set of commands and behavior, conditions and actions should not be hard-coded into the programming environment. Rather, each device should be able to plug its own set of
commands into the language. Therefore, during designing the architecture of the language, I considered the conditions and actions as plugins that are loaded into the programming environment dynamically. Each plugin is a file that contains the information
about the condition/action including its visual representation that is defined in HTML
and its execution behavior that is defined as a JavaScript method, that is invoked by
6
http://mindstorms.lego.com/
Here is a short video of the visual programming environment in its early stages:
http://www.youtube.com/watch?v=TyvblzUptwI
7
42
3.4 User Interface
Figure 3.3. JavaScript-based visual rule-based programming language was created
to rapidly compose the behavior of Lego MindStorms NXT.
the interpreter during execution.
We succeeded in controlling a Lego Mindstorms NXT from a Web page. The details of this research including the architecture and development of the programming
language are reported in [6].
3.4 User Interface
Following the feasibility phase, I developed AgentWeb, an educational game design
environment, using open-Web technologies8 . Being built using open-Web technologies, i.e., HTML5, JavaScript, and CSS, AgentWeb transcends the need to install any
software on the desktop. Users design, store and play the games in the cloud which
eliminates the barrier of uploading/downloading games from/to the desktop.9
AgentWeb’s game design and programming model is inspired by AgentSheets, an
end-user authoring tool for creating agent-based computer games and scientific simulations [101]. Therefore, similar to AgentSheets, users are capable of creating 2D
games and scientific simulations such as Sokoban, Pac Man, Space Invaders, Frogger,
and Game of Life with AgentWeb.
AgentWeb provides users with a visual Integrated Development Environment (IDE)
8
A proof-of-concept is online at: https://agentweb.inf.usi.ch
The current version of AgentWeb runs in Google Chrome, Mozilla FireFox and Apple Safari browsers,
which together count for more than 60% of worldwide browser usage share according to the statistics
provided by StatCounter as of April 2012.
9
43
3.5 Architecture
to create the game characters as agents, draw their graphical depictions, program
them, and place them together in the game scene, all from inside the Web browser.
Figure 3.4 shows the AgentWeb IDE used for developing a Frogger game. The IDE
consists of four main panes:
• Agent Gallery maintains the game objects as a list of agents and their assigned depictions. Each agent may have several depictions, possibly representing different
states of the agent. Agents typically represent game characters.
• Depiction Editor lets users draw depictions for the agents10 . Depictions are graphical representations of agents.
• Programming Environment lets users program the agents using a visual programming language.
• Scene Editor, known as worksheet in the IDE, lets users create the game scene
by drawing the game objects on a HTML5 Canvas. The user can create several
scenes for a game.
The IDE presents two different working environments between which the user can
switch at anytime. The first environment, as shown in Figure 3.4, hosts all four panes
at the same time and gives users an interactive programming environment. The second
environment provides a full screen view of each pane and lets the game designer focus
on one activity at a time (i.e., depiction editing, programming, or scene editing) while
accommodating more development area in the environment.
3.5 Architecture
AgentWeb has a model-view-controller (MVC) architecture [49]. The view, i.e., user
interface, is implemented as HTML and CSS widgets which are loaded dynamically into
the Document Object Model (DOM) tree. The controller is a set of JavaScript objects
that manipulate the model according to user interaction with the interface. The view
is updated corresponding to the changes of the model.
Nowadays, interactive Web applications are commonly built using a JavaScript
framework used for developing an interactive user interface that connects to a data
model on the server through AJAX. AJAX provides a protocol that allows the clientside to continue interacting with the user while the server updates the model and the
view simultaneously. In AgentWeb, however, we dismissed the choice of AJAX because
the latency imposed by the network causes the user interface to not meet the needs of
direct manipulation and interactive programming features. Therefore, to minimize the
communication with the server, we built AgentWeb completely as a client-side game
10
AgentWeb
employs
an
open
(http://code.google.com/p/paintweb/)
source
open-Web
image
editor
called
PaintWeb
44
3.5 Architecture
Agent
Gallery
Scene
Editor
Depiction
Editor
Conditions
Programming Environment
Actions
Figure 3.4. The AgentWeb IDE, used by novice programmers to develop open-Web
games inside the browser.
design environment. Accordingly, the data model resides on the client side, skipping
the network communication.
Benefiting from client-side MVC architecture, AgentWeb exposes a direct manipulation environment to the game designer. For instance, as soon as an agent depiction
is modified in the editor, the modification is instantly applied in the other panes including in the programming environment and the scene editor, even while the game is
running. To enable interactive programming, the game’s program written in the visual
language is interpreted, or compiled as described later, at the client side. Therefore,
as soon as the user modifies the agent program, the code is recompiled and the user
receives immediate feedback in the game.
Figure 3.5 shows the activity diagram of creating an agent, programming it, instantiating it, and executing it in AgentWeb. The execution process follows an MVC
architecture also. The agents are created in the agent gallery. Each agent carries its
own visual program which is written by the game designer and compiled to JavaScript.
Instances of agents are created in the runtime system by the scene editor. The runtime
system executes the program of agent instances repeatedly and visualizes the agent
45
3.5 Architecture
user creates
an agent
[erase instance]
user selects
the agent
[scene editor]
user
edits the
scene
program
the agent
or edit the
scene?
[programming
environment]
[add instance]
user programs
the agent
user draws an
agent in the
game scene
visual
program
User Interface
(View)
Business Logic
(Controller)
game character
functionality
compositions
compiler
translates to
JavaScript
instantiate the
agent
JavaScript
executable
Runtime System
(Model)
runtime agent
game engine
executes the
agents
graphics engine
renders the output
in the browser
Figure 3.5. The overview of AgentWeb architecture: life-cycle of agents, user interaction with the system, and flow of operations during the execution process are
shown.
instances according to their current position and depiction in the browser.
Agent instances are created either at design time through the scene editor or at runtime by other agents. Scene editor includes a toolbox that lets users create instances
of the selected agent in the agent gallery or erase agent instances from the scene. Further tools are provided for multiple creation/erasing of agent instances. Each agent
instance inherits its program and depictions from one of the agents created by the
game designer. The inheritance is implemented through JavaScript prototypes. During
the instantiation, an object consisting of the game character functionality is instantiated and associated with the agent instance. The game character object maintains the
run-time state of the agent, such as current position and depiction in the scene, and
provides methods to define the behavior of the agent, e.g., moving the agent, changing
46
3.6 Visual Programming Language
the depiction of the agent, or sending messages to other agents.
3.6 Visual Programming Language
AgentWeb supports Web users with a visual rule-based programming language environment. The language is an instantiation of the language used by AgentSheets [105].
Each agent contains its own program that consists of a set of methods. Each method
is either handling an event defined in the system, i.e., mouse click, or a user-defined
method which is called by other methods. The desired functionality of each method is
composed of ‘rules’ as the basic construct of the language that enables decision making.
For the sake of simplicity for the end users there is no loop construct. However, each
agent consists of a Forever method that is executed repeatedly at runtime. Each rule is
a composition of conditions and actions. The user composes methods, rules, conditions
and actions visually, by means of drag-and-drop, to achieve the desired functionality.
Figure 3.6 shows the control flow of a method at run time. When a method is triggered
at runtime, the rules are tested sequentially and the first rule with all of its conditions
being satisfied will be executed. Figure 3.4, for instance, shows the program of the frog
agent in a Frogger game. The user defines that if the down arrow key is pressed and
the cell below the frog is a floor cell and the frog is alive, then move the frog down.
According to the classification of Burnett and Baker [21], the design and development of the visual programming language of AgentWeb falls into the following categories:
• VPL-I. Environments and Tools for VPLs
• VPL-II.A.11: Language Classifications 7→ Paradigms 7→ Rule-based Languages
• VPL-II.B.2: Language Classifications 7→ Visual Representations 7→ Iconic Languages
• VPL-III.A.2: Language Features 7→ Abstraction 7→ Procedural Abstraction
• VPL-III.B,E: Language Features 7→ Control Flow, Event Handling
• VPL-IV.B,C,D: Language Implementation Issues 7→ Efficiency, Parsing, Translators
(interpreters and compilers)
3.6.1 Language Specification
At the finest grain, the visual language grammar has a type system consisting of “visual
data types" such as direction, depiction, or a number. Direction, for instance, can let
users visually choose one of nine possible cells around an agent (including the agent’s
47
3.7 Client-side Execution
next rule
exists?
[No]
[Yes]
[No]
is the
condition
satisfied?
execute the
condition
conditions
and
actions
[Yes]
[Yes]
next
condition
exists?
[No]
execute
all the actions
Figure 3.6. The control flow of a method in an agent program.
own cell). As another example, depiction lets the user choose one of the defined depictions of an agent. The language consists of 10 visual data types as specified in
Appendix A.
The set of language instructions (conditions and actions) form a domain-specific
language for creating computer games. Each instruction provides a piece of functionality that has zero or more input arguments. Each input argument has a type from the
set of visual data types. For example, as shown in Figure 3.4, the see condition has
two input arguments. The first argument is a direction and the second argument is a
depiction. Figure 3.7 shows the main conditions and actions of AgentWeb. The full language consists of 28 instructions, including 12 conditions and 16 actions as specified
in Appendix A.
3.7 Client-side Execution
The execution process follows the MVC architecture of AgentWeb:
• View: The agents are created in the agent gallery. Each agent carries its own
visual program which is composed by the game designer.
48
3.7 Client-side Execution
Conditions
Actions
Figure 3.7. Main conditions and actions of AgentWeb
• Controller: Instances of agents are created in the runtime system by the scene
editor.
• Model: Once the agents have been programmed and instantiated in the game
scene, the game must be executed. The runtime system executes the program of
agent instances repeatedly and notifies the view to visualize the agent instances
according to their current position and depiction in the browser.
In order to execute the agents inside the browser, a client side interpreter or compiler translates the visual program into JavaScript which is then executed by the AgentWeb runtime system. Initially, we implemented an interpreter to translate the visual
program to JavaScript. However, due to not achieving the required performance in
the games, we designed and implemented a compiler to replace the interpreter, as
explained in Section 3.8.1. The game design environment is capable of switching between interpretation and compilation at any time.
AgentWeb includes a runtime system that executes the agent instances and renders them in the game scene. Figure 3.8 shows the runtime system components and
their relationship. The runtime system is composed of agent instances, game engine, user input handler, and graphics engine. The game engine maintains the list
of agent instances and executes them, receives the user input, e.g., mouse clicks and
keystrokes, and applies them during the execution. The graphics engine renders the
updated agents in the browser. As the figure shows, each agent instance inherits its
49
3.8 Performance Issues
agent
depictions
visualProgram
compiledMethods
executeCompiledMethod
gameCharacter
currentDepiction
position
change
key
move
see
....
1
1
agentInstance
gameCharacter
execute
0..n
1
gameEngine
agentInstances
graphicsEngine
userInput
runningLoop
1
1
graphicsEngine
canvasElement
renderScene
1
1
inputHandler
keyboardBuffer
mouseBuffer
recordInput
Figure 3.8. Runtime system components and their relationship.
program and depictions from one of the agents created by the game designer. The
inheritance is implemented through JavaScript prototypes. During the instantiation,
an object consisting of the game character functionality is instantiated and associated
with the agent instance. The game character object maintains the run-time state of the
agent, as described before.
Algorithm 1 shows the pseudo-code of the execution cycle. The game engine contains an iterative execution cycle called ‘runningLoop’, which executes all the agents
repeatedly as long as the scene is running. The user can stop or continue the execution
at any time. After each execution cycle, the graphics engine renders the game scene
according to the updated position and depiction of agents. The agents are rendered
in a HTML5 <canvas> element which provides native graphics drawing in the Web
browser.
3.8 Performance Issues
The initial implementation of the runtime system had a major defect: more sophisticated games were slow, not reaching the interactivity level they required to be playable!
Therefore, I profiled the game execution process in order to uncover any bottlenecks.
Two major sections in which time was spent were the interpretation of user’s visual
program and Canvas drawing calls. To address the first bottleneck, I implemented a
50
3.8 Performance Issues
Algorithm 1 Game engine execution cycle
1:
2:
3:
4:
5:
6:
7:
loop
for all agent instances a in the scene do
execute a
end for
apply the user input (e.g., mouse and keyboard)
render the scene
end loop
compiler to avoid the overhead of repeatedly interpreting the visual program; to address the second, I developed a differential rendering algorithm to reduce the number
of calls to the Canvas API. The details of code compilation and the differential rendering algorithm are described in the rest of the section. The achieved performance
improvements are reported in Section 3.9.
3.8.1 Code Compilation
Every time a user-created program is executed by the runtime system, the interpreter
traverses the visual program, which is represented as a tree, to translate it to JavaScript.
The tree traversal and converting the visual representations to their equivalent executable JavaScript is a costly process. The compiler, instead, traverses the visual
program tree only once and serializes it into a string containing a JavaScript function
that is equivalent to what the interpreter would execute if it were to traverse the tree.
The JavaScript function is then injected to the browser engine using the eval function.
From now on, the compiled JavaScript function will be called by the game engine during the execution. Furthermore, by generating a JavaScript function and pushing it to
the browser engine, rather than interpreting the methods, instruction by instruction,
the generated code will have a chance to get optimized by the browser engine. This
is in particular effective in JavaScript compilers such as Chrome V8, which compiles
the entire code to machine code, and JIT compilers such as Firefox JaegerMonkey and
Safari Nitro in which hot spots are detected and compiled to machine code at runtime.
It is important to note that any performance optimization must keep account of the entire Web execution stack. In particular, browsers are performing an increasing number
of optimizations. Any optimizations performed by the developer should not duplicate
or, worse, hinder, such optimizations.
To preserve the interactive programming feature in AgentWeb, the compilation process is performed completely transparently to the game designer, i.e., in the user interface there is no button to save/apply the changes made in the program. To preserve
the transparency, the code translation is performed dynamically and incrementally, i.e.,
every single change in the visual program triggers the compilation process. To reduce
the compilation time, compilation is only performed for the changed method of the
51
3.9 Performance Evaluation
agent, rather than the whole program of the agent. As soon as the method is recompiled the agent hot-swaps the newly compiled code. Hot-swapping involves deleting
the old compiled method and injecting the new code into the browser engine. By
leveraging dynamic client-side incremental compilation, with no server communication involved, I achieve low compilation and response times that are imperceptible to
the game designer.
3.8.2 Differential Scene Rendering
The initial graphics engine was rendering all the agents on the scene at the end of
each execution cycle. By profiling the graphics engine I found out that most of the
time is spent in calls to the Canvas drawImage function. On the other hand, I found
out that the number of actual graphics changes of the game at each execution cycle is
quite low. Therefore, I designed a differential rendering algorithm which updates only
those parts of the scene that have been updated during the last execution cycle. The
differential rendering algorithm only re-draws the game characters that either have
moved or changed their depiction, thus reducing Canvas drawing costs. Algorithm 2
describes the pseudo-code of differential rendering. Section 3.9 shows the performance
gain of using the differential rendering algorithm compared to rendering the whole
scene.
Algorithm 2 Graphics engine differential scene rendering
1:
2:
3:
4:
5:
6:
7:
for all cells c in the scene do
if c was modified in the last execution cycle then
for all agent a positioned in c do
redraw a in <canvas>
end for
end if
end for
3.9 Performance Evaluation
The performance of games developed in AgentWeb is lower than the individually designed games developed by professional programmers due to the high abstraction level
of our game design model necessary to support novice programmers:
• AgentWeb’s domain-specific language constructs, i.e., conditions and actions,
such as see and move are translated to several JavaScript instructions that involve
calls to game and graphics engines. Hand-coded routines for one-off games can
be shorter than the automatically generated code.
52
3.9 Performance Evaluation
• Professional developers rely on performance profiling to detect the costly routines and optimize them individually. In the case of AgentWeb, I do not expect
novice programmers to perform such activities. Rather, I have to ensure that the
runtime system—consisting of the game engine and graphics engine—is highly
optimized to execute different types of games responsively.
Accordingly, I have been continuously profiling the interpreter, compiler, translated
code and runtime system during the development of AgentWeb to improve the performance of games. In fact, early profiling of the runtime system led to the development
of compiler and differential rendering algorithm as described in Section 3.8. To measure the execution performance of complex processes in the runtime system, I had to
develop our own profiler, rather than relying on the browser’s built-in profiler, to ensure that the collected performance data is comparable across different platforms and
browsers. The profiler instruments the main components of the runtime system including both the game and graphics engines to measure the time spent in different phases
of the execution process.
I implemented the instrumentation with the goal of minimizing the overhead on
execution and graphics performance:
• During running the evaluations I avoided any function calls to the profiler for
collecting and analyzing the data, as the overhead of function calls in a repetitive
execution process can increase the execution times. Instead, I defined some
profiling variables with global scope to profile the execution such as the number
of calls to the game and graphics engines.
• I hard-coded the profiling variables into the game and graphics engines instead
of turning them on or off using an if switch
• I deferred all the processing of collected data to after completing the execution
of the test cases as explained in the rest of this section.
In this section, I compare the performance of three computer games and one mathematical simulation, game of life. The games are instances of typical arcade games created using existing end-user programming environments. I included the game of life
as an instance of compute- and graphics-intensive simulation compared to games. The
games and their properties are sorted in ascending order of computation and graphics intensiveness in Table 3.2, i.e., Sokoban is the least compute- or graphics-intensive
game.
I conducted the evaluation across the latest versions of major desktop Web browsers
that are capable of running AgentWeb, i.e., Chrome 15 with V8 engine, FireFox 7.0 with
JaegerMonkey, and Safari 5.1 with Nitro11 . All the browsers were freshly started and
11
Currently, AgentWeb is not compatible with Internet Explorer and Opera.
53
3.9 Performance Evaluation
Game
Runtime
Scene Size
Depiction Size
Screen Size
Agents
(cells)
(pixels)
(pixels)
Sokoban
86
13x9
32x32
416x288
Frogger
∼290
20x13
32x32
640x416
Space Invaders
∼325
15x17
32x32
480x544
Game of Life
1024
32x32
32x32
1024x1024
Table 3.2. Evaluated Games and Their Properties
only one tab was opened during the performance evaluation. Desktop-based evaluations were performed using a MacBook Pro with 2.66 GHz Core 2 Duo processor and
4GB of RAM on two operating systems: Mac OS X 10.7 and Windows 7 running on
VMware Fusion 3.1. I also evaluated the performance of games on two smart devices:
Atomic browser on iPad 2 running iOS5 and FireFox on Samsung Galaxy S II running
Android 2.3. 12 AgentWeb is not usable on smart devices due to the lack of support
for touch interactions. However, being built using open-Web technologies, games are
executable in their browser as long as they support JavaScript and HTML5 canvas.
To find out whether games created in AgentWeb meet the interactivity required by
computer games, I measured the display rate, which is the common metric to measure
the responsiveness of interactive applications including computer games [123]. By
today’s standards, 30 FPS is the acceptability threshold for playing almost any game.
However, the required display rate depends on the genre of the game. 30 FPS is
required by action games in which the player tracks animated objects. Such strict
requirements do not apply to less interactive games such as board and card games. In
AgentWeb, the display rate is equivalent to the number of cycles executed by the game
engine per second, i.e., Algorithm 1, as at every cycle a complete interaction process
with the user takes place, including applying the user input and rendering the scene in
the browser.
For each game, I ran the evaluation in four different configurations in each browser
to compare the performance of code interpretation versus code compilation and complete scene rendering versus differential scene rendering.
• N: Code interpretation with complete scene rendering (no optimization)
• D: Code interpretation with differential scene rendering
12
Though AgentWeb runs in older versions of mobile Safari, in iOS5, mobile Safari crashes while loading AgentWeb due to an unknown bug. Therefore, I employed a popular substitute, Atomic browser,
which uses native iOS UIWebView to render Web pages. Atomic browser has lower performance than
mobile Safari as Nitro engine is only available to Safari.
54
3.9 Performance Evaluation
• C: Code compilation with complete scene rendering
• CD: Code compilation with differential scene rendering (most optimized configuration)
With each configuration, the test executes a snapshot of the game with a fixed
starting point in the game scene for at least 30 seconds and 50 execution cycles. Each
execution was repeated 30 times to achieve statistically valid results. The mean and
95% confidence interval values were computed for all the measured parameters. The
computed confidence intervals are very small and therefore neglected.
Figure 3.9 (a,b,c, and d) shows the display rate for the games across all the browsers.
For each game in each browser, the four columns correspond to the four described configurations in order from left to right. The performance gained by the compiler and
differential rendering (C, D, and CD) is obvious. With the most optimized configuration (CD), user-created games are playable across all the tested platforms. Specifically,
the games execute fast enough to be playable on smart devices. Even in the case of
Game of life, as a more extreme application compared to typical games, the achieved
frame rates on desktop browsers are reasonably high.
As shown in Figure 3.9, the performance gain of the compiler over the interpreter
reveals that the more compute-intensive the game is, i.e., more translated code is executed by the runtime system, the higher the performance gains will be. On the other
hand, the more graphics-intensive the game is, the more the differential rendering
behavior will be closer to the complete rendering and therefore the lower the performance gain. The performance gains of differential rendering are more pronounced
when used with the compiler (CD vs. D) than when used with the interpreter (D vs.
N). This is due to the higher frame rate when executing compiled code, which causes
a higher number of calls to the graphics engine.
Despite the differences of the achieved performance across different browsers, the
observed results demonstrate that the state-of-the-art browsers render the Web as a
hospitable platform for running not only one-off highly optimized games but also enduser created games built on higher levels of abstractions. Moreover, using the combination of compilation and differential drawing techniques (CD) makes it unlikely for end
users to reach the performance limit of AgentWeb on desktop browsers. User-created
games execute reasonably fast also on existing smart devices. Benefiting from relying
on the open Web, the games run on these devices independently of their hardware and
operating system specifications.
To explore possible relationships between the achieved results and existing browser
benchmarking suites such as SunSpider13 and V814 , I measured the performance of
employed browsers using these suites. The results are shown in Figure 3.9 (e, f).
The general performance of games may be predicted based on these benchmarks, e.g.,
13
14
http://www.webkit.org/perf/sunspider/sunspider.html
http://v8.googlecode.com/svn/data/benchmarks
55
3.9 Performance Evaluation
b) Frogger
a) Sokoban
Display Rate (FPS)
240
Display Rate (FPS)
240
210
210
180
180
150
150
120
120
90
90
60
60
30
30
0
0
Chrome Chrome FireFox FireFox
OS X
Win 7
OS X
Win 7
Safari
OS X
Safari
Win 7
Chrome Chrome FireFox FireFox
OS X
Win 7
OS X
Win 7
Atomic FireFox
iPad 2 Galaxy
c) Space Invaders
Display Rate (FPS)
240
Display Rate (FPS)
240
210
210
180
180
150
150
120
120
90
90
60
60
30
30
0
Safari
OS X
Safari
Win 7
Atomic FireFox
iPad 2 Galaxy
Safari
Win 7
Atomic FireFox
iPad 2 Galaxy
Safari
Win 7
Atomic FireFox
iPad 2 Galaxy
d) Game of Life
0
Chrome Chrome FireFox FireFox Safari
OS X
Win 7
OS X
Win 7
OS X
Safari
Win 7
Atomic FireFox
iPad 2 Galaxy
Chrome Chrome FireFox FireFox
OS X
Win 7
OS X
Win 7
Safari
OS X
f) V8 Benchmarking Suite
e) SunSpider Benchmarking Suite
Score
Time (ms)
8000
9000
7000
8000
6000
7000
6000
5000
5000
4000
4000
3000
3000
2000
2000
1000
1000
0
0
Chrome Chrome FireFox FireFox
OS X
Win 7
OS X
Win 7
Safari
OS X
Safari
Win 7
Atomic FireFox
iPad 2 Galaxy
Chrome Chrome FireFox FireFox
OS X
Win 7
OS X
Win 7
Safari
OS X
Figure 3.9. On the left side (a,b,c,d), the display rate is measured from execution
of games in a number of browsers. Each browser has been benchmarked with four
different configurations. Configurations are ordered from the slowest with no optimization (N) on the left to the fastest with code compilation and differential scene
rendering on the right (CD). Evaluations with the result of less than 3 frames per
second are not visible; On the right side (e,f), the performance of the same browsers
are measured using two browser benchmarking suites: e) In SunSpider, the faster
browser has the lower benchmarking time. f) In V8, the faster browser has the
higher score.
56
3.10 Usability Testing of the Game Design Environment
browsers with better benchmarking results are, in general, also capable of running
the games faster. However, finding a straight correlation between the performance
of games and those of algorithms benchmarked by these suites is hard to prove and
out of the scope of my work. This suggests that low-level benchmarking of browsers
is not suitable for estimating the performance of interactive games. A more sophisticated benchmarking suite with game and graphics rendering components may be more
suitable to estimate the performance of games precisely.
3.10 Usability Testing of the Game Design Environment
AgentWeb’s game design model has been inspired by AgentSheets, which has also
influenced other game design models [61]. AgentSheets has been widely used in
schools and computer clubs to teach computational thinking and programming to middle school and college students [12]. By employing AgentSheets’ game design model,
I am using a model that has enabled novice programmers to learn and build computer
games. However, AgentWeb has been designed using Web technologies, and therefore
its user interface has a somewhat different layout which has been built according to
state-of-the-art Web application development practices. Therefore, the usability of the
Web-based environment must be verified.
During the development phase, I continuously tested the usability of AgentWeb
through several game design sessions in an after-school computer class in a middle
school in Boulder, CO, with different participants. In each session, one or two middle
school students familiar with AgentSheets were asked to use AgentWeb with no prior
training. Familiarity with AgentSheets let me skip the training process and instantly
find out whether (i) participants are comfortable with the Web application layout; and
(ii) participants are actually able to create a game with AgentWeb.
In the school settings, participants were asked to create a basic version of a game
of their own interest such as Frogger, Pac-Man, or Space Invaders in a 30-60 minutes
session. I observed that in all cases, the participants were capable of learning the new
layout and creating a basic game. However, as the AgentWeb layout is different from
AgentSheets, initially participants required some time to adapt themselves to the new
interface. Participants were asked to think aloud during the session but they were
not provided with any support. While participants were comfortable with the general
layout of AgentWeb, our observations led to improving the user interface, including
re-arranging several buttons, adding shortcuts in the programming environment, e.g.,
for creating rules and methods, overriding the backspace button to avoid accidental
navigation to the back page, and adding snippets of visual programs to assist the game
designer.
After improving the user interface, I tested the environment with five end users who
had never programmed before. They were guided through the IDE for 10-15 minutes,
with the goal of familiarizing them with the IDE layout, image editor, programming
57
3.11 Conclusion
Agent Gallery&"
Image Editor%"
Scene Editor$"
Programming
Environment#"
!"
!"
#!"
$!"
%!"
&!"
'!"
(!"
!"#$%&#"'()$*+%
Figure 3.10. A sample usage pattern automatically recorded from end user’s interaction with Web-based AgentSheets . X axis is the passage of time in minutes. Y
axis is the section in which user has spent the time. The time periods of less than
10 seconds are considered as noise and removed.
environment, and the scene drawing and execution features. After the training period,
they followed a tutorial on how to make a simple Frogger game consisting of creating
5 agents, editing their depiction, programming, and drawing them in the game scene.
They all managed to finish the tutorial with a mean time of 58 minutes, excluding
training time, which in practice is about the same time dedicated in the game design
course in the middle schools to finish the Frogger tutorial using AgentSheets.
Benefiting from running in the cloud, I collected the participants’ use of the game
design. The AgentWeb’s layout manager was instrumented to record the user interaction with the user interface including focus on different IDE panes, e.g., the scene
editor, based on the mouse movements. The recorded data was transparently sent to
the server using AJAX and saved in a database. Analyzing the collected information
from the users led to discovering new flows which were extrapolated from the user
interaction with AgentWeb. For instance, Figure 3.10 reveals that the user tends to
edit the depiction of an agent right after creating the agent, while spending time in
the scene editor right after programming. I used this information to improve the IDE
layout: to keep the position of image editor and agent gallery closer together, and the
programming environment and the scene editor next to each other.
3.11 Conclusion
In this chapter, I posed the challenges of bringing educational game design to the Web
in order to make game design a cooperative learning activity. I presented the AgentWeb
game design environment and its engineering to address the challenges imposed by
the open-Web platform. AgentWeb adopts a well-known (rule-based, agent-oriented)
game design model to overcome usability issues, uses open-Web technologies to ensure
58
3.11 Conclusion
portability across platforms, and executes entirely on the client side to avoid potential
network latencies. AgentWeb employs client-side code compilation to JavaScript and
optimized rendering algorithms to achieve playability of the games across desktop and
mobile browsers. While existing game design environments may support creating more
interesting 3D games and games that require a physics engine, my focus is to teach
computational thinking in a cyberlearning platform, not necessarily the building of
visually appealing games.
In the next chapters, I will describe how I created a cooperative platform around
the game design environment that I presented in this chapter. The ultimate goal of
this cyberlearning platform is to extend the learning of computational thinking outside
of the classroom and scale it up to the much larger community of Web users. The
ecosystem will support collaboration and mentoring among the learners.
Chapter 4
Transition to the Web as a
Cyberlearning Platform
With AgentWeb, novices benefit from a cyberlearning platform for educational game
design, as discussed in Chapter 1.Technologically, we scale up the access to educational
game design world-wide through the Web platform. All users need to get started and
gain computational skills is a Web browser.
However, ‘teaching’ computing concepts through game design does not scale up
just by using a cyberlearning platform. In going to the Web, the main challenge is to
shift the educational practices from a traditional classroom environment to a cyberlearning platform. In traditional classroom environments, problem-based learning is
employed in which the teacher guides the students to solve the design and computational problems during the game design session. In the absence of a teacher, lack of
support for learning design and computational thinking makes these tools less motivating and less useful specially for newcomers. Therefore, even though some of the
existing game design environments are freely available to use, these environments are
mostly used in classrooms and are rarely used by a novice user with no access to a
teacher. I have identified two major problems with existing approaches:
• Although existing game design environments are rich for practicing computing
concepts, teaching problem solving skills, such as computational thinking [147]
and design thinking [52], is not explicitly supported by these tools and therefore
it is left to the teacher.
• Educational materials in the form of video tutorials, wikis and sharing Websites
mostly focus on teaching the features of the game design environment rather
than teaching the design and computational thinking skills. Step-by-step tutorials guide novices to create existing games, which may not be meaningful to them
[82].
As a result, novices who are not explicitly taught design and computational thinking
59
60
4.1 A process to support design and computational thinking
fail to use the game design environments properly and to achieve the expected skills.
During this research, I have explored ways to compensate for the lack of a teacher
when scaling up the computer science education using cyberlearning, to take full advantage of the Web as a world-wide teaching platform rather than just as a mean to
deliver software. I envisioned two solutions to compensate for the lack of a classroom
environment:
• Identify the most common computational and design thinking problems and provide tools and techniques to support novices with immediate feedback.
• Enable computer-supported cooperation among users and support peer learning
mechanisms
In this chapter, I describe my efforts to design and implement the first solution.
Chapter 5 will deal with the second solution. In Section 4.1, I describe the design and
evaluation of a game design process for learners which takes into account the absence
of the teacher. In Section 4.2 I analyze the results of evaluating the process, and suggest
solutions to integrate this game design process into the game design environment as
a software. Section 4.3 concludes. Some parts of this chapter have been published in
[3].
4.1 A process to support design and computational thinking
Based on my experience with novices during game design sessions with AgentSheets
and AgentWeb, I identified three common thinking tasks that novices repeatedly have
to accomplish to build a game. In this section, I present the design of a game design process, and its supporting tools, to compensate for the lack of a teacher. By
structuring the thinking problems, the process can assist the students in solving design
and computational problems during using AgentWeb. The benefits of this approach
include:
• Users gain computational and design thinking skills directly from the game design environment.
• In the cyberlearning settings, users receive support from the environment in the
absence of a teacher.
• The game design process will be personally meaningful to the students as they
build these skills implicitly while being actively involved in building a game
based on their own story.
The proposed solution provides support to novices in informal learning contexts
while remaining complementary to formal learning contexts to assist the teachers dur-
61
4.1 A process to support design and computational thinking
ing the game design course1 .
4.1.1 Identified Computational and Design Thinking Activities
Similar to other game design environments, design and computational thinking activities are not explicitly supported in AgentWeb. I identified three problems most
common to all the game design scenarios of AgentWeb that require design and computational thinking:
• Determining the game objects;
• Determining the behavior of a game object;
• Figuring out how to program a behavior.
To support the students with solving these problems during the game design, I
developed a solution for each of these problems. Our only assumption is that the
student is capable of expressing the game story in a written form. I use the Frogger
game as the example throughout this section. Here is a Frogger story [109]:
You are a frog. Your task is simple: hop across a busy highway, dodging
cars and trucks, until you get the to the edge of a river, where you must
keep yourself from drowning by crossing safely to your grotto at the top of
the screen by leaping across the backs of turtles and logs. You will die if
you get hit by trucks and cars!
The mentioned problems can be solved by analyzing the game story as follows:
Determining the game object
In order to determine the game objects, the user will be asked to mark all the words
appearing as a noun in the game story. Each noun is most likely one of the game
objects. For each noun, the user should determine if it plays a role or appears in the
game scene. If so, it will be a game object. In the above story, the nouns are: frog,
task, highway, car, truck, edge, river, etc. As an instance, task does not appear in the
game scene and therefore it is not a game object.
Determining the behavior of game object
The user is asked to mark all the verbs in the game story that are related to a game
object. Each verb defines a behavior, i.e., a piece of functionality, that requires programming. For instance, the frog hops, dodges, drowns, crosses, leaps, dies, and gets
hit.
1
In my opinion, at least at this time, cyberlearning is complementary to the traditional classroom
practices, to teach computing concepts.
62
4.1 A process to support design and computational thinking
Programming the behavior
To support the novice programmers to program the behavior, they will be given a list of
possible functionalities, each of which is to be related with one or more verbs. The common behaviors of game objects that are used across several games are identified and
knowns as computational thinking patterns in the context of AgentSheets [56]. Computational thinking patterns provide a higher level of abstraction than programming
that can be used to support novices in identifying how to program a certain behavior.
Similar patterns can be re-used for AgentWeb as it implements a game design model
similar to AgentSheets. Accordingly, the user will be asked to match the determined
behavior with a corresponding computational thinking pattern. For instance, hopping,
dodging, crossing and leaping are instances of cursor control pattern, while getting
hit is an instance of the collision pattern. For each computational thinking pattern,
the user will be shown the corresponding visual program that should be written in the
programming environment.
By providing support for solving these problems, users will be able to explore the
game design environment based on their own game stories in the absence of a teacher.
As a complementary solution to formal learning settings, students can be given a list
of stories to choose from according to their interest. The teacher can lead the students
to solve the mentioned problems according to their chosen story.
4.1.2 Evaluation
I conducted an evaluation to assess the feasibility of incorporating the designed solutions for computational thinking problems into the game design process. Considering
that introducing problem solving steps increases the complexity of the game design
process, I evaluated the following hypothesis:
Combining the support for computational thinking with the tool usage results in a game design process that is easy enough for the students to learn,
specifically in the absence of a teacher.
Evaluation goals
In order to evaluate our hypothesis, I developed a basic game design process which
combines design and computational thinking problems with steps to use the game
design environment. The process is tailored for AgentWeb, as shown in Figure 4.1.
Each step belongs to one of two types: computational thinking or tool usage. To
operationalize the term “easy to learn” in the hypothesis, the following aspects were
defined to be evaluated:
• Usability: each step of the process should be straightforward and easy enough to
be accomplished by a novice programmer.
63
4.1 A process to support design and computational thinking
Game Story
1: Determine the game agents
2: Create agents
3: Draw the agent on the scene
4: Determine the behavior of agent
5: Match the behavior to the corresponding
computational thinking pattern
6: Use the selected computational thinking
pattern to program the behavior
7: Test the program
Computational Thinking Step
Tool Usage Step
Figure 4.1. Evaluated game design process combines the computational thinking
tasks with tool usage tasks.
• Learnability: assuming that the user has accomplished the process before, whether
or not with the support of a teacher, it should become easier to repeat the process.
• Self-directedness: users are able to accomplish the steps in the absence of a
teacher.
Design of the evaluation
The process was developed as a proof of concept in order to validate our hypothesis.
I opted for an expert evaluation in order to collect comments and suggestions from
teachers to be used for revising the process and game design environment prior to a
64
4.1 A process to support design and computational thinking
user evaluation.
I adopted the cognitive jogthrough [115] process, a lightweight usability testing
method, for the evaluation session, in order to involve teachers to provide feedback
on the process as much as possible. By using cognitive jogthrough I focused on the
usability of the process while designing a game. Because the process is divided into
several steps, cognitive jogthrough as a task-oriented evaluation process lets us evaluate each step independently. I extended the jogthrough questionnaire to evaluate the
learnability and self-directedness of the process along with the usability (See Appendix
B).
Conducting the evaluation
I invited four high-school teachers to evaluate the game design process. Three teachers
were teaching students in the last two years of high school and one teacher in the first
year of high school. The evaluation was conducted in a single two-hour session. First,
teachers were shown a 6 minute video of how to create a game using AgentWeb. The
goals of evaluating the developed process were explained to the teachers.
Nine tasks were evaluated in the context of Frogger, in order to show the order and
relevance of the tasks in the process. Each task corresponded to one step of the game
design process. The programming tasks (corresponding to steps 5 and 6 of the process)
were repeated twice to implement (i) moving the frog using arrow keys (cursor control
pattern) (ii) the frog getting hit by the truck (collision pattern). Each task was classified
as either computational thinking or tool usage, according to its corresponding game
design step. For each task six questions were asked, out of which four were directly
assessing the usability, one for learnability and one for self-directedness. I used the
cognitive jogthrough questionnaire both for computational thinking and tool usage
steps with a slight tailoring of evaluation questions for each step.
The tasks were performed by the moderator and teachers filled out the questionnaire for each task. As the computational thinking tasks are not implemented in the
game design environment, I used examples to demonstrate how to solve the problem.
To match a behavior with the corresponding computational thinking pattern in step 5
I printed a list of computational thinking patterns on the paper to be used by teachers
as the reference (See Appendix B).
4.1.3 Results
Teachers rated each research question based on cognitive jogthrough scaling: 0 = no
user; 1 = some users; 2 = half of the users; and 3 = most users. Figure 4.2 compares
the average rate of computational thinking tasks to tool usage tasks on each evaluation
aspect. All the average values fall between 2 (half of the users) and 3 (most users),
which, together with the teacher comments, shows that the teachers are persuaded that
all the steps are easy to be learned and to be accomplished by high-school students.
65
4.2 Comments from Teachers: Towards Smarter GDEs
Computa<onal"Thinking"Tasks"
Tool"Usage"Tasks"
Average'Rate'by'teachers'
3"
2"
1"
0"
Usability"
Learnability"
Self4directedness"
Figure 4.2. Mean of teachers’ rating of research questions based on cognitive jogthrough’s scaling.
Teachers commented that the game design environment is intuitive to accomplish the
designed tasks. The solutions to design and the computational thinking tasks are clear
and well-related to the the tool usage.
Tool usage tasks gained higher average rate than the computational thinking tasks
in all the aspects. Teachers explicitly pointed out that the computational thinking solutions should be integrated into the game design environment to be useful to students.
For instance, to support step 5, one of the teachers explicitly suggested to develop a
wizard tool to assist the students in matching the behavior to the corresponding pattern. Teachers considered the game design process to be complementary to be used
both inside the classroom and to give assignments to students to use at home. To support the students at home they suggested to use video tutorials on how to accomplish
computational thinking steps. The comments from the teachers are elaborated further
in Section 4.2.
4.2 Comments from Teachers: Towards Smarter GDEs
The teachers were positive about how intuitive the user interface is and that students
are comfortable with Web applications and virtual environments; hence, there will
be no initial barrier to engage students in the game design activities. The modified
jogthrough questionnaire that we used to collect the comments focused on the phenomena of using a GDE in the absence of a teacher, which became the main theme of
66
4.2 Comments from Teachers: Towards Smarter GDEs
the comments.
The evaluation shows that accomplishing the computational thinking steps is more
complex than tool usage. Teachers suggested that the support for computational thinking steps should be integrated into the game design environment, rather than only provided through exercises and tutorials as complementary teaching material to students.
In this section, I reflect upon the comments and offer ideas about how a GDE can provide better support to novices, by means of smart software tools and user interfaces, in
the absence of a teacher. The suggested solutions are based on my observations with
how novices use AgentWeb.
In response to the comments about seeding the environment with tutorials and premade materials, I later added these to the environment. For computational and design
thinking challenges I have drawn a solution to add a software tool inside the game
design environment to support novices. However, further exploration of developing
and evaluating such software was left as future work. Last but not least, I focus on one
of the issues which is related to this thesis and explore it in further details, which is to
enable cooperation among game designers.
4.2.1 Examples, Tutorials
The teachers state several times in the comments that practice is essential for learning. They mentioned that once students have accomplished each step once, repeating
them will become easy. One teacher commented on Step 6: "Who has managed to
accomplish this task more or less, will also succeed in a similar one." Another teacher
commented on Step 3 says: "Once an example have been created it will become much
easier to use these tools." Their comments suggest that each step of the process is
well-defined and repeating the steps becomes easier by practice.
The teachers stated that having tutorials and examples is necessary to engage students in learning activities from their home. On Step 1, a teacher stated:“If the teacher
knows the environment and is able to explain it, there will not be any problem. If
the students approach the platform independently (from home) a video like the one
you showed us will be useful. Interface seems quite intuitive." This led me to create a
‘Tutorials’ section on the AgentWeb Website and seed it with several new tutorials, as I
will explain in the next chapter.
On-Demand Educational Materials
A teacher commented on Step 6: "Tutorials should present examples for each pattern,
otherwise some students will not be able to accomplish the Step." This is In-line-with
my argument at the beginning of this chapter regarding the video tutorials of existing
GDEs to be too generic and focusing on how the GDE is used rather than how to solve
the computational and design problems that novices face.
67
4.2 Comments from Teachers: Towards Smarter GDEs
In particular, the teachers commented about having on-demand tutorials about
each computational pattern, and conditions and actions that are used in them. This
is mainly because initially users are not familiar with the behavior of the provided
condition and actions, and how they can be composed together to implement a computational thinking pattern. In other game design environments such as AgentSheets
and Scratch, a help document with some examples is provided inside the GDE to describe the behavior of each programming construct. In AgentWeb, I opted for creating on-demand video tutorials for each condition/action. On-demand video tutorials
demonstrate the implementation of one or more computational thinking patterns that
use the corresponding condition or action. All the videos have the same theme of interaction between two agents of circular shape, one in blue and one in pink, called
Mario and Maria respectively. I developed these videos only for those programming
conditions and actions that are used in Chapter 6.
Templates and Galleries
One teacher commented on Step 2: "It could be interesting to prepare some icons
with images ready made and in first instance start from those for creating an agent."
Accordingly, I added templates from which users can start a game. Currently, the
templates contain two pre-made games, an implementation of a Frogger, and a Ball
game that is used in the experiment reported in Chapter 6. More games can be easily
added to the list of templates.
Another teacher commented on Step 3:"Maybe adding a library where some simple
characters are already available that can be used immediately." I marked this as a future
work.
4.2.2 Computational Thinking Issues
The teachers commented that Steps 4 and 5 are most difficult as “they need thinking
and abstraction, rather than taking actions." The dependency of Step 5 on Step 4 —
they should be able to determine the behavior properly so that they can match it to
a program — requires knowing a list of possibilities and examples. Therefore, more
examples are required. One teacher stated on Step 5: "Without video/tutorials to
explain the models [,i.e., patterns,] to be used for the comparison [,i.e., matching,]
while providing examples it would be difficult for students to learn by themselves.
Once the reasons beyond running a comparison [,i.e., matching,] are clarified, this
step will be almost trivial."
The teachers mentioned that students will gain expertise gradually, as long as the
process is clear enough and well-supported in the user interface: "The more they use
the program the more the actions necessary for the implementation become obvious."
One teacher noted that by introducing Steps 4 and 5 into the game design process,
which separates the complexity of abstract thinking from using the programming tools
68
4.2 Comments from Teachers: Towards Smarter GDEs
in Step 6, learning step 6 becomes easier: "If the previous task [Step 5] is obvious,
then the work to complete this task [Step 6] will be easy. In order to run it at home, it
is necessary to have performed Step 5." However, another teacher commented on the
importance of teaching the control flow of the language (in Step 6): "Also in this case
there will be some more student experiencing difficulty. It is all about linking properly
cause/effect that are not always self explanatory and evident."
Tool Support for Computational Thinking
One teacher commented on Step 1: "Non-appropriate logic/linguist knowledge/background could create difficulties in some students." In order to support novices in Steps
1, 4 and 5, building proper tools with an easy-to-learn UI becomes crucial, so that we
minimize confusion and provide examples and support right at the time they require
it. It is important that such UI do not become restrictive and do not get in the way of
using the game design environment. Moreover, by introducing additional user interfaces, we have to make sure that features of the programming environment, such as
Turing-completion, are not compromised. Two of the possible enhancements are:
• Identifying Nouns and Verbs: Natural language processing techniques such as
part-of-speech tagging can be used to process the game story and identify the
nouns and verbs to support the user [137]. The immediate feedback returned
by such computer assistance will also help the user learn how to solve these
problems.
• Matching Computational Thinking Pattern with Expected Behavior: A basic user
interface can be developed inside the game design environment to guide the
user through added game design steps. For instance, user interface elements,
e.g., dialogs, can be added to let the user enter the game story, identify the
nouns to create the game objects, identify the actions from the verbs used in
the story, and match the identified behavior with corresponding computational
thinking patterns. As suggested by teachers, a wizard tool can assist the user in
choosing the right computational thinking pattern.
4.2.3 Design Thinking Issues
The teachers pointed out issues that are related to design thinking, rather than computational thinking.
Behavior Specification
One teacher stated about how students can realize that Step 6 is completed: “For somebody it will still be possible to neglect some details. For example the fact that the frog
could walk back.” This comment points to the issue of software specification. Verbs
69
4.2 Comments from Teachers: Towards Smarter GDEs
give us clues about the general behavior of the object, but to specify a behavior programmatically requires further effort. However, novices are not professional software
engineers. Therefore, following the spirit of educational game design, we do not want
to introduce a specification language to specify the behavior of game objects (unless
the language is created for novices!). Therefore, it sounds more reasonable to explore
other solutions.
I suggest to introduce a shared repository that is made of the ways that users currently have specified such behaviors. Information retrieval techniques can be used
to mine the behaviors out of existing games and use them as examples to help students figure out what the possibilities are and choose the behavior that is closer to
what they think. As a consequence, other novices can reuse these pieces of programs
in their games. Automatic detection of such patterns is currently being explored for
AgentSheets [64].
Decomposition, Cardinality
A teacher commented on Step 3: "Some clear instructions have to be provided, for
example it has to be spelled out that there is only one object frog, the road can be
repeated and it is necessary to point out the differences between tools that enable users
to achieve these goals." The comment points to the importance of design thinking. In
this case cardinality is an issue. At first sight, we need one big road in the game. But,
partially being forced by the grid-based game design model of AgentWeb, this would
not be possible. Therefore, the road should be decomposed to ‘road tiles’. The road tiles
are then repeated on the scene. How could we teach concepts such as decomposition
and cardinality, specially if there is no teacher involved?
4.2.4 Software Engineering Process
The teachers stated several times that students always forget something and figure it
out along the way, therefore they will need to go back and fix it:
• Step 4: “after reading the text will be able to abstract the behaviour of all various
characters. They will easily forget something and will need to come back to this
task."
• Step 4: “The abstract nature of the task implies that students while completing
it will easily forget some of the character behaviors. The students will spot that
later on while trying to accomplish next tasks and they will then have to complete
the description for the missing behaviors."
• Step 4:“There is a risk that, because of a lack of patience, some element could be
neglected. The problem could emerge in following phases and could create the
right occasion for discussing software engineering.”
70
4.3 Conclusions
• Step 6:“there will be always some students forgetting the details."
This suggests that the game design process, as carried out by novices, will be highly
iterative, even for the simplest tasks. We can not expect the students to follow a formal
step-by-step process.
On the other hand, students are very much reward-oriented, which is to get to see
the game running. One teacher commented on Step 7: "The test action is obvious
and it is the most fun part for the student." Therefore, to keep the students motivated,
it would be better to have small iterations which get the student to see the effect in
the game scene. As I discussed in the previous chapter, I have employed Interactive
Programming to fill the gap between Steps 6 and 7 by giving immediate feedback on
the behavior of game objects while students are programming them; thus shortening
the path from programming to testing. In a sense, interactive programming incorporates the fun of testing the behavior into the programming step. This could be further
extended by employing ‘Conversational Programming’ techniques [102], which during
executing the game it visualizes those programming constructs that are being executed
and shows the result of their execution. For example, users can see the result of the
evaluation of each condition at runtime, by changing the color of condition to green if
it is true or red if it is false.
4.3 Conclusions
In this chapter, I described a game design process that takes into account the absence of
a teacher in a cyberlearning platform and how software tools around it can bring further support to the learner. Affected by the design of the process that separates thinking steps from practical steps, the evaluation of the process by the teachers brought
up several issues related to software engineering, design thinking, and computational
thinking which students would struggle to accomplish if they are left on their own,
while teachers considered the sole programming task to be trivial and gained by little practice. To me, this raises a question: would it not be the time for educational
game design environments to shift the focus from practicing programming to practicing software design? The answer may be found in the end-user software engineering
community.
The software may go a long way to support the user, but at the end having a
teacher or peer to specifically guide the learner throughout the design and computational thinking issues is crucial. The next chapter explains the design and development
of a community-oriented Website which enables peers to cooperate around game design activities.
Chapter 5
Building a Cooperative Game Design
WebSite
By going to the Web, ties to a physical location are broken, which is the first step
towards expanding educational game design. However, because educational game
design involves learning activities, lack of a teacher on the Web should be compensated
for. During the evaluation presented in the previous chapter, a teacher pointed out that
"[by moving outside of the classroom,] it is not possible to intervene on specific issues".
In several steps of the process, at least one teacher commented on the fact that having
help from another person can be useful to accomplish the task:
• On Step 1: "With a minimum possible external support problem solving will
become easy enough for the majority of tools to be used."
• On Step 2: "Some students would need external help (teacher and peers, classmates) in order to complete this task."
• On Step 4: "It is not easy starting from the story script to imagine all possible
actions and modalities necessary in order to perform these actions.", and continues: "I believe it is necessary that an example is provided and shown to student
entirely with the teacher’s help."
• On completion of Step 5: “They will miss the contribution of an expert that could
help at each critical point.”.
Their comment reinforced my initial ideas that by going to the Web we can benefit
from building a world-wide community of learners/teachers following the successful
stories such as Wikipedia. In this chapter I describe the design and implementation
of a cooperative Website around AgentWeb to support the exchange of knowledge and
building communities of practice outside of the classroom environment and in the
absence of a formal teacher. Accordingly, students will be able to collaborate in game
71
72
5.1 Issues with existing models of cooperation
design activities and learn from each other. Ideally, such a community becomes a selfsustaining ecosystem in which the lack of an instructor is compensated by skilled peers
[66].
In the rest of this chapter, in Section 5.1, I argue about the inadequacy of existing supplementary game design Websites to trigger collaboration among learners or
to move the learning practices to cyberspace. In Section 5.2, I present a vision for
enabling cooperation and learning which is centered on the game design activities. In
Section 5.3, I describe the development of such a cooperative Website around AgentWeb. In Section 5.4 I describe the development of educational materials for the Website. In Section 5.5 I present the usability testing of the Website through a workshop
with 13 high-school students. Section 5.6 concludes.
5.1 Issues with existing models of cooperation
Sharing is the foundation of collaboration and the building block of a cyberlearning
platform. It should be intrinsic to the learning process and available to users at no
cost. As discussed In Section 2.2.3, some of the existing educational game design environments provide support for sharing the creations through a supplementary Website.
Users upload their creations on the Website in order to share them with others. The
others can play the game from inside the browser, but they need to download the
games to their machine in order to browse or modify the inside of the game, including
its graphic design and program. In case the user modifies the game and would like to
share it again with others, she should upload the game back on the Website. Issues:
• From the usability point of view, the upload-download process often takes more
than 10 seconds, which distracts the user from her flow of thoughts [89].
• The sharing model provided by existing environments lets users share the result
of their design, which is a game, i.e., ‘what’ they have built, not the design process, i.e., ‘how’ it was built. Such a sharing model draws a line between playing
the game as the entertaining artifact and exploring the game components as the
educational artifact. Rather, I believe that a cyberlearning environment around
game design activities should allow sharing and collaboration of the design of
the games. Downloading the games to the desktop to explore the inside of the
game disconnects the user’s flow of activities on the Website.
• As the game design process takes place on the desktop of individuals, rather
than in the knowledge sharing environment, these Web sites fail in mediating
knowledge exchange among peers to learn how to solve computational and programming problems. As a result, the comments on the game page appear to be
mostly praises about the game and there is very little information about ‘how’ the
73
5.2 A Vision for Unobtrusive Cooperation
game was built. This can be easily observed on the large game sharing communities such as Scratch 1 . Instead, the game design discussions mostly take place
on the forum of the Website.
The upload-download process is the first barrier to motivating cooperation and
peer learning among users. In line with this observation, recent studies on users of
Scratch with the largest game and animation sharing community shows that while the
social skills of Scratch users increase through using the application, there is no clear
results in increasing collaboration among scratchers [118]. I believe that one reason
that causes this ineffectiveness is that sharing is provided as an extrinsic feature to the
game design process which requires upload-download steps.
5.2 A Vision for Unobtrusive Cooperation
AgentWeb has been built as a Web application in order to easily integrate it into a Web
platform that motivates cooperation and social learning (see Section 3.1.2). In the
big picture, a browser-based game design tool is a stepping-stone towards integrating
the benefits of Web-based cooperative tools into the educational game design. Now
that I have the game design tool in the browser, I can build the required features that a
community of learners need around the tool. In Section 2.4.2, I described how the Web
and its participative culture have come into attention as a cyberlearning platform. Such
a cyberlearning platform transforms the traditional educational game design practices
in the following ways:
• From individual to cooperative game design: rather than running the GDE on
the desktop of learners, the game design activities take place at the center of
the networked platform which is designed for education, i.e., a cyberlearning
platform.
• From upload-download to unobtrusive exploration: as the game design takes
place online, sharing the games and their design with others happens with no
effort from the user. The games are being played online as before. But more
importantly, access to the design and program of the game is just one click away.
The platform encourages the learners to explore the inside of the games they play
and look for the solutions for their games in other games with similar behavior.
• From the desktop to the cloud: by going to the Web, the benefits of softwareas-a-service are added to the game design experience. First, learners can access
their game design workspace from any device and any platform, as long as they
are connected to the Internet2 By having their workspace in the cloud, users do
1
scratch.mit.edu
There are technical solutions to run a Web-based application even when the user is offline. Therefore,
being connected to the Internet is not a hard constraint.
2
74
5.3 Implementation
not need to maintain the project files and carry them on a disk back and forth
between the classroom and home.
• Community-oriented learning: Communication tools such as chat and chatroom
are integrated into the platform to add the social dimension to the game design
practices. The communication tools can be further integrated with social networking applications such as Facebook and Twitter. This will allow learners to
quickly reach a community of game design practitioners to find a solution for
their game design problems. In addition to using a forum to search for generic
problems, most of the discussions can be situated in the context of the game a
user is creating right when the user is creating it.
• Opportunity for understanding the learning process of novices: As the game
design takes place in the cloud, all the information on user interaction with the
game design environment can be collected such as the types of problems they
face, the usability issues with the tool, the amount of time they spend on each
part of the environment, etc. Together with the chat logs, this information can
be analyzed to understand the learners’ progression and how to support them
better.
Table 5.1 summarizes how the Web transforms the game design practices of the
learner from a traditional school environment, which I call Game Design 1.0, to a
community of practice using a cyberlearning platform, which I call Game Design 2.0.
5.3 Implementation
According to the discussed vision, I created a Website around the game design environment that facilitates cooperation among game designers in several ways. The Website
brings together Web 2.0 best practices and comments from the teachers during the
evaluation that was presented in the previous chapter. The Website consists of the
following components:
• Personal Workspace: the Website provides each user with a personal online
workspace to manage their own creations. The design and code of games are
stored—in the cloud—and shared by users, enabling collaboration among users
on the design of the game. Users can explore the profile of other users and the
games that they have created.
• Community Building: unlike the existing game sharing portals, in which every
game can be modified by its owner only, the AgentWeb Website allows other
users than the owner of a game to join as a designer of that game, and upon
approval of the game owner they will have access to modify and save the new
75
5.3 Implementation
Game Design 1.0
Game Design 2.0
Solitary game design
⇒
Cooperative game design
Download, Modify, Upload
⇒
Instant sharing
Share the play
⇒
Share the design
Play the game online,
⇒
Explore the design of
edit the game on the desktop
Different instances of the GDE
the game and learn online
⇒
on each computer
Seamless access to one cyberlearning
platform from different devices
Learner maintains project files
⇒
Platform maintains project files
External communication tools
⇒
Integrated communication tools
Support through forums
⇒
Situated game design rooms
Learning in the classroom
⇒
Social learning in an online community
from the teacher
No data on patterns of
tool usage or collaboration
from peers and existing games
⇒
Easily recording tool usage and chat logs
on the server
Table 5.1. Comparison of Game Design 1.0 and Game Design 2.0 styles
version of the game. As a result, each game can have a community of game
designers who will work on the same game but at different times.
• Unobtrusive sharing and exploration of the design: the Website exposes all the
games created by all users and provides search and categorization functionalities. Users can browse and play all the games created on the Website, explore the
design and program of each game while still playing, and even modify the game
and re-run it to see how it works. As a result, the Website becomes an opensource community for game designers, as opposed to existing desktop-based
game design environments which only let users share games as a closed-source
software, unless they explicitly download the game.
• Integration with social networking applications: the Website provides basic integration with Facebook, the dominant social networking application, which eases
the user’s access to the Website—they do not need to create yet another new
account. Users can register and login to the AgentWeb Website using their Facebook account. I plan on expanding the social networking scenarios of the Website
further, to allow users to share and collaborate within their own social network.
76
5.4 Content Development
As a result, AgentWeb will be better integrated to the social media lifestyle of
students.
• Asynchronous communication: popular asynchronous participation channels including comments, rates, and like buttons were added to each game to allow users
to express their feedback to the designers of a game.
• Instant messaging and chatrooms: the teachers suggested to add synchronous
communication tools to support students in seeking help and information from
other members quickly. Accordingly, we added both instant messaging and chatroom features to the Website. While instant messaging allows students to collaborate with each other in a one-to-one manner, chatroom supports the communication at the community level. The chat facility is available on all the pages
of the Website—as an overlay—which lets users cooperate at any moment, in
particular when designing a game.
• Forum: the forum can be used by users to ask questions which can be seen and
responded to by all the members of the Website. Furthermore, the forum serves
as the knowledge base of the Website in which users can search previously posted
problems and answers.
5.4 Content Development
To help novices to get up and running with AgentWeb, the Website required help content. I added several video tutorials, two forums, and four showcase games to the
Website.
5.4.1 Video Tutorials
I created two sets of video tutorials:
• How to use AgentWeb: includes ‘Introduction to AgentWeb’, ’Creating a Game’,
and 5 programming tutorials which, all together, are 76 minutes long. Because
there were several programming concepts to teach, I first wrote a manuscript for
the programming tutorials, to make sure that the tutorials cover all the concepts
that novices need to learn. Appendix C contains a list of all programming tutorials and the programming concepts that are taught in them, together with their
manuscript.
• Programming instructions (conditions and actions): I created one short tutorial
for each of five conditions and five actions, all together 23:20 long, which are
loaded from inside the game environment. These tutorials were made following
the discussion in Section 4.2.1 for those conditions and actions which are used
in the evaluation that is described in Chapter 6.
77
5.5 Usability Evaluation of the Website
5.4.2 Forum
I created two forums and added several questions to each forum:
• FAQ: contains several questions about what agentweb is and whether users need
to install any software to use it.
• Programming Questions: contains several programming questions, all of which
start by “How do I ... ." Each question contains a response pointing to one of the
video tutorials of the programming instructions.
5.4.3 Showcase
I re-created four of the games that were originally created with AgentSheets, each one
containing a number of computational thinking patterns:
• Frogger: Absorb, Collision, Generate, Transport
• Sokoban: Pull, Push
• Space Invaders: Absorb, Collision, Generate, Script
• Game of Life: Perceive/act synchronization
The purpose of these games is two-fold:
• Showing what kind of games can be created with AgentWeb.
• Using them in the future evaluations,as used in the next chapter, as a way to find
out how a certain computational thinking pattern is implemented.
5.5 Usability Evaluation of the Website
To study the usability of the Website, I conducted a one-day workshop with 13 highschool students in the age range of 15 and 20. Participants were given 90 minutes of
hands-on tutorial on how to create a frogger game. Next, they were given 60 minutes
to improve their games. The workshop setting was a round-table in a classroom. A
total of 40 students had originally applied to take part in the workshop and among
them only 13 were selected. The selection was done by the teachers at the school
without my involvement. I have little information about the selection criteria. I was
told that the teacher thought these students would benefit from the workshop. After
a brief introduction, each student was asked to fill a brief questionnaire about age,
education and preferred subjects, from which it emerged that the majority was inclined
towards science, mathematics and Informatics. The students showed their interest and
dedication by engaging deeply equally during the tutorial part and even more so while
78
5.5 Usability Evaluation of the Website
Strongly Agree5"
Agree4"
Not Sure3"
Computer
Use
Game Design
Environment
Cool
Annoying
Easy to learn
Entertaining
Confusing
Entertaining
Confusing
Cool
Annoying
Easy to learn
Strongly Disagree1"
Fun
Interesting
Frustrating
Boring
Disagree2"
Cooperative
Website
Figure 5.1. Mode of participants’ rating based on the likert scale.
working on their games; they kept working even during breaks. At the end of the
workshop participants were asked to fill out a questionnaire regarding their overall
game design experience. Figure 5.1 shows the mean user experience measured using
likert scale in three aspects: use of the computer during the workshop, the game design
environment, and the cooperative Website.
Comments on the game environment were mixed: there was enthusiasm on how
much they could build in such an easy way, one comment described the environment as
phenomenal, a tribute to how simple and effective it proved to be. However, those students who already had experience with regular programming languages commented
on how the environment was too simple and not challenging enough for them. This
is in line with the fact that the environment has been designed and targeted to novice
users.
All students commented on how easy and fun the system was and a third of them
mentioned how much they enjoyed the possibility to share their work with their peers.
This is in line with their behavior during the game design session. Even if students
did not ask many questions directly from tutors and limited their active participation
during the workshop—taking an active role in a class and showing genuine interest
in a topic is seen as showing off and not ‘cool’—still, more than half of them enjoyed
publishing their earlier work during development, showing how much they liked to
share their new achievements with their peers and valuing this option.
As the students were collocated and the number of participants was low I can not
state any statistically valid conclusions. However, I observed that most of the partic-
79
5.6 Conclusion
ipants naturally cooperated with their adjacent participants to solve their own problems. Even though all the participants were physically in the same location, a few used
chat to seek help from other participants. Students could see the advantages of sharing their work as a direct way to start a collaboration and get help while establishing
themselves and let their game represent them.
5.6 Conclusion
In this section, I presented the design, implementation, and evaluation of a Website
around the AgentWeb game design environment. The Website is a proof-of-concept cyberlearning platform based on software-as-a-service and Web 2.0 practices. I discussed
how such a platform addresses the inefficiencies of existing game sharing models for
educational game design environments. Some parts of this chapter have been published in [2].
In the next chapter, I conduct a more comprehensive study with novices using
AgentWeb in a “real” cyberlearning setting, i.e., accessing the Website from their home
and with no access to a formal teacher. This will help me understand how novices
employ the provided tools of the Website to find solutions to their computational problems and whether educational game design practices are replicable in a cyberlearning
platform.
80
5.6 Conclusion
Chapter 6
Experimenting with Cooperation
In previous chapters we described the design, development, and evaluation of AgentWeb, a cooperative educational game design environment built using open-Web technologies and instrumented with collaborative features according to Web 2.0 practices.
AgentWeb was built to evaluate my thesis on whether we can broaden participation in
educational game design by moving it to the cyberlearning space. As I showed in the
previous chapters, building a working prototype was done successfully. However, the
question still remains:
can educational practices be replicated in a cyberlearning platform, in which
the physical classroom and formal learning is replaced by a cyberlearning
platform and online communication tools?
In this chapter, I report on an experiment which I conducted in order to understand
the function of computer-supported cooperation in the computational thinking activities
taken place in a cyberlearning educational game design environment. Understanding
the function of cooperation is the central phenomenon in this experiment. Computational thinking defines the dependent variables, i.e., the variables of interest are those
related to how a computational problem is solved. Therefore, in this experiment I
rule out other tests including those related to usability, e.g., how users draw the game
scene, or navigate through the Website. As for computational thinking, the focus of
this experiment is how users solve computational problems, rather than how they learn
programming.
The Web is the cyberlearning platform on which I rely for running AgentWeb and
the Website around it. Computational thinking activities take place through a Web
browser and educational activities are supported through the Website’s features including the forum, instant messaging, and chatroom.
In the rest of this section, in Section 6.1, I define the set of questions that are the
subject of this experiment. In Section 6.2, I briefly explain the experiment’s process.
In Section 6.3, I describe how the data is collected during this experiment. In Section
81
82
6.1 Research Questions
6.4, I explain the procedure of conducting the experiment. In Section 6.5, I present
and discuss the results. Section 6.6 concludes by answering the research questions.
6.1 Research Questions
The core theme of the research questions of this experiment is to find out whether
with the provided ways for cooperation, novices can practice and gain computational
thinking skills.
1. a) Do novices rely on cooperation channels to solve their problems or they prefer
to find the solution individually? b) Which cooperation channels do users employ
most in order to solve the problems?
2. How do cooperative tools affect novices’ performance? Can they solve more
computational problems if they cooperate?
3. Do novices learn computational thinking concepts by engaging in game design
activities in a cyberlearning platform?
6.2 Research Method Overview
I used a mixed methods design, i.e., employing both quantitative and qualitative methods, for this experiment. The main part of the experiment is developed as a quantitative
evaluation. Qualitative evaluation is developed as complementary in order to support
the findings of the quantitative research.
Each participant is involved in a game design session in which they are asked to
solve a number of computational thinking tasks. The performance of participants, such
as the number of problems they solve and whether they use cooperation tools to solve
the problems, is collected during the game design session.
After the game design session, participants are asked to reflect on their experience
with the use of cooperation tools during the game design session. The collected data
will be used to explore the function of cooperation tools and explain the measured
quantitative data.
A computational thinking quiz measures whether participants can identify those
computational thinking patterns that they implemented during the game design session. Furthermore, a basic pre-post test is developed to measure the change of the
users’ opinions about using a Web-based software, and computational thinking skills.
6.3 Data Collection Methods
A number of questionnaires were developed to measure different aspects of the user’s
participation during this experiment, as will be discussed throughout this chapter.
83
6.3 Data Collection Methods
• Registration form (Appendix D.1) collects demographic information and the participant’s prior computer science and programming knowledge.
• Pre-test questionnaire (Appendix D.2) collects the participants’ computer and
Web usage habits including Web applications and communication applications
(D.2.1), gaming experience (D.2.2), social networking (D.2.3), preferences regarding desktop versus Web applications (D.2.4), and their interest in computer
science (D.2.5). The last two parts are measured again after the experiment.
• Task completion questionnaire (Appendix D.3) collects the users’ use of cooperation tools after each task.
• Computational thinking quiz (Appendix D.4) measures the users’ capability to
identify computational thinking patterns.
• Qualitative questionnaire (Appendix D.5) collects the users’ opinion about the
use of cooperation tools, tutorials, and the game design session in general.
6.3.1 Computer Logs
A logging system was added to the Website and the game design environment to log
the following data in a database on the server.
• The page the user is visiting
• Every minute that the user spends in the game design environment
• Each time the user explores the inside of a game, including the information of
the game.
• Whether the user is playing, designing, or exploring a game
• The game design part (scene editor, programming environment, image editor) in
which the user is spending time, the switches between parts and the time spent
in each part. I disabled the interactive programming layout so that at each point
in time I know with which part of the game design environment (scene editor,
programming environment, image editor) the user is interacting.
• Programming actions including drag-and-drop, delete, new rule, new method,
duplicate, and modification to the values of the programming conditions and
actions.
• Video tutorials watched by the user.
• The idle time was recorded if the user did not interact with the environment (using mouse or keyboard) for more than 20 seconds, unless the user was watching
a video tutorial.
84
6.4 Procedure
• Each time the user leaves or returns to the browser page by going to another
browser tab or application.
• The chat and chatroom conversations were also recorded with a time stamp in
the database.
6.4 Procedure
6.4.1 Sample
Recruiment was done mainly through Craigslist and Amazon Mechanical Turk. All the
interested individuals were asked to fill out the registration form which served as pool
from which I selected a sample of participants who met the following constraints:
• Participant must be a novice programmer with no programming background.
• Since I focused on the Gen-Y population, the participants’ age must be between
16 and 30.
I invited 97 participants who met these constraints. Out of those who responded, 9
(4 male, 5 female) participated till the end of the experiment. The participants were:
min=18, max=29, mean=23.6 years old. 7 participants were from 4 different states
of the US, and 2 others were from Russia.
The registration form also served as the demographic questionnaire by collecting
the participants’ age, gender, location including the state, the country they went to
school, proficiency in the English language, profession, education, and computer skills
level including whether they have ever taken a computer course, written a program or
created a website, and the programming languages they have used to do so.
6.4.2 Pre Test
All the participants were asked to take the following steps prior to attending the game
design session:
• Create an account on the AgentWeb Website and watch AgentWeb’s two introductory video tutorials. This would let the participants decide whether they are
interested in participating in this experiment.
• Fill out a questionnaire (Appendix D.2.1-D.2.3) including i) computer and Web
usage habits. I collected this data to understand the participants’ background to
be able to better support them during the game design session, and be able to
explain the users’ performance and detect possible outliers in future correlational
85
6.4 Procedure
studies; ii) Pre-test questions (Appendix D.2.4-D.2.5) collect the participants’ beliefs and preferences regarding the difference between desktop and Web applications and their opinions about computer science and computational thinking,
as described below.
• Install the Google Chrome browser to make sure that the experience will be the
same among all participants.
A pre-post questionnaire was used (Appendix D.2.4-D.2.5) to measure the change
of the participants’ opinion in the following:
• A 16-items questionnaire based on likert scale was developed to measure the
user preferences in using a Web application versus a desktop application.
• 8 questions were created to measure the interest of the participants in taking
an introductory CS course, their opinion about the effect of computer science
knowledge in their daily life, their opinion about whether learning computer
science/programming is a solitary or collaborative activity, the context in which
they would prefer to learn computer science, and whether they think they can
learn a professional programming language.
6.4.3 Game Design Session
A 2:30 hours long game design session was scheduled for each participant. Prior to
the session, they were asked to watch 3 out of 5 programming tutorials which all together take about 34 minutes.During the session, participants were asked to work on 5
computational thinking tasks (see Section 6.4.3). To focus on these tasks, I left out the
solving of design problems such as identifying the agents and their actions. Therefore,
I created a ‘ball’ game template containing the agents. I specified each task in detail to
lower misunderstanding of the tasks. Moreover, I removed the conditions and actions
that are not necessary to accomplish the tasks from the programming environment.
I integrated the tasks into the UI of AgentWeb so that participants could easily
read them without navigating to another page or opening a browser tab on the side.
Moreover, to measure how participants spend time on each task, I added buttons in the
tasks to record starting, pausing, and completing the tasks on the server (see Figure
6.1).
After completing each task, whether they completed the task successfully or not,
the participants were taken to a short questionnaire which collects the information
about the communication tools they used during the task (Appendix D.3).
Moderation and Help
I moderated the session to make sure that participants go through the tasks one by
one and report the start and completion of the tasks properly, and filled out the task
86
6.4 Procedure
Figure 6.1. The participant receives the description of tasks from inside the game
design environment. The logging system records the flow of completing the tasks
and the time spent on each task.
completion questionnaire after each task. During solving the tasks, I was remotely
tracking the activities of the participants by:
• Exploring the inside of their game, simply by going to their game page and pressing the explore button, i.e., a feature of the Website that I described in Chapter
5. Accordingly, I could follow their progress in implementing the solution.
• Monitoring the logs that are recorded in the database to see how they interact
with the environment, whether they watch tutorials, or explore other games to
find a solution.
• Taking notes on each task about the kind of problem that the participant is facing
and their solution to the problem. I used these notes during analyzing the results.
As a moderator, I provided help in finding the solution to the tasks in an organized
way, with a set of principles:
• Prior and during the session, I asked the participants to ask for help in case they
do not understand the tasks or can not figure out the solution to the task.
• I did not offer any help with the tasks unless they asked for help.
• I did not have any way to modify their game nor its program. Therefore, if they
could not find a solution, they would give up and continue to the next task.
In case they asked for help, I made sure that I understood the problem that they
were facing, by asking them, and in most cases by tracking their activities. It was not
possible, at least for me, to arrange a structured protocol for how much should I help
them as the demands of participants were varied. However, my golden rule was to
offer the minimal help to guide them towards the right solution, without mentioning the
right solution, and I let them find their own way toward it. The following may be a
rough order of how I guided them:
87
6.4 Procedure
• If there was a tutorial with an answer to their problem, I would point them to
the tutorial.
• If participants were not sure which condition or action they should use. I suggested them to explore the video tutorials of conditions and actions that were
embedded into the game design environment.
• I asked them questions about what they wanted to do and how they think it
should be done. This helped them to clarify their ideas and correct themselves.
• I explained to them the meaning of the program that they had written. This
helped them to understand what the problem was.
• If they still did not understand what the problem was, I explained the problem
to them.
• If they were still struggling with no success, I suggested them to explore some of
the similar games to find a solution.
• In a few cases, I pointed them to a certain game to explore.
Tasks
The tasks should satisfy the following predicates:
• The tasks should be representative of users’ interaction with AgentWeb. However, I leave out the usability testing tasks as they have been done to some extent
in previous chapters.
• If an active community of users exists around educational game design with
AgentWeb, the tasks should show us what kind of cooperation is triggered between users.
• For educational reasons, the tasks should help us measure the learning outcomes.
Therefore, tasks with no or little learning outcomes, such as drawing graphics or
the game scene are left out.
I considered two options for designing the tasks:
• Programming: Participants are given some programming tasks to learn the programming language of AgentWeb. This will include tasks such as creating rules
and methods, understanding the order of execution of the rules, etc. The programming tutorials that I created would serve as the basis for creating a set of
tasks.
88
6.4 Procedure
• Computational thinking patterns: Participants are given a set of behaviors for a
number of agents in the context of a game, and they would have to figure out a
solution to implement those behaviors.
To compare these two approaches, teaching programming is a bottom-up approach
in which the learner would learn the programming constructs without having a representative list of problems in the domain, while teaching computational thinking patterns is a top-down approach which starts by exploring the problem first and gain the
programming skills in the course of solving the problem. I opted for computational
thinking patterns, as discussed in Section 2.3.1, as the basis for designing the tasks:
• Computational thinking patterns are the building blocks of creating games. These
patterns occur across different games repeatedly. They are a good representative
of problems in the game design domain.
• There is a correlation between learning a pattern in one game and being able
to repeat them in other games. Based on this, I have created a computational
thinking quiz to measure the participants’ learning after the game design session.
• If the participants are engaged in only learning the programming constructs, it
would be hard to conclude about their ability to use their programming skills to
solve the actual problems in the game design domain. Moreover, assessing their
understanding of programming will become either long or incomplete.
As the Website already provides a number of games in the showcase, which users
have had the chance to explore even before the game session starts, I wanted to make
sure that the tasks are represented in the context of a novel game which is different
from those games. Therefore, I designed a new game, a ‘Ball’ game with 5 tasks
corresponding to 5 major computational thinking patterns. The ball game consists of
the following agents: Player, Ball, Goal Net, Goalie, Field Center. The computational
thinking patterns, their corresponding task, and the programming conditions/actions
that are used to implement those tasks are:
1. Keyboard/Cursor Control: User moves the player using arrow keys [Key, Move]
2. Push: Player moves the ball [Key, See, Make, Move]
3. Absorption: If goalie sees the ball on its right, it removes the ball [See, Erase]
4. Generation: When user presses the X button, the Field Center generates a ball
on its top [Key, New]
5. Collision: When the ball reaches the Net, the status bar shows a GOAL! message
[Stacked, Show Message]
89
6.5 Results
6.4.4 Post test
Computational Thinking Quiz
After completing all the tasks, the participant was asked to answer a quiz which measures their learning of computational thinking tasks that they implemented during the
game design session (Appendix D.4). The computational thinking quiz consists of 10
items, 2 items per each computational thinking pattern, each pattern corresponding
to one of the 5 tasks. One of the two items for each pattern shows the behavior of
one or two agents, including the visual program of the agents. The item then asks the
participant to choose the one out of four choices that best describes the behavior of
the agents. To avoid random selection and increase the precision of the measurement,
I introduced a second question for each pattern in which the participant has to select
the best term that describes the behavior shown in the first question. The terms are
the names of computational thinking patterns.
Post-test questionnaire
The post-test questionnaire repeats the same questions of pre-test questionnaire (Appendix D.2.4-D.2.5) to measure possible changes in the participants’ opinions about
using a Web-based application, and their interest in computer science.
Collaboration Questionnaire
The final questionnaire that participants filled out is a qualitative questionnaire including open questions about the effect of cooperation tools, the effect of tutorials, and
their opinion about the game design session (Appendix D.5).
6.5 Results
The experiment was conducted with 9 participants. I merged the chat logs with the
logs of participant activities and went through them to precisely mark the beginning
and the end of working on each task, to remove the communications and activities of
the participant before and after each task. While marking the beginning of each task
was straightforward, marking the end of the task was not straightforward from the
logs. I used signs from the chat and computer logs to mark the first logged event that
shows that the participant has realized that she has accomplished the task as the end of
the task.
In total, participantes attempted to solve 41 tasks. Table 6.1 shows the performance
of the participants in detail, according to the tasks.
90
6.5 Results
Task
Sample
Size
Full
Solution
Partial
Solution
Minor
Help
Major
Help
Q1
Q2
Time
Spent
(mean)
Tab
Switches
(mean)
Program
Actions
(mean)
Chat
Messages
(mean)
1
9
9
0
0
3
9
9
24:00
16
41
39
2
8
5
2
2
6
5
6
1:13:58
56
151
69
3
8
8
0
3
2
8
7
27:55
29
29
26
4
8
7
1
1
2
8
8
15:27
7
13
16
5
8
8
0
2
2
7
8
16:19
20
24
18
Total
41
37
3
8
15
37
38
μ: 31:32
μ: 25
μ: 52
μ: 34
90%
7%
20%
37%
Percentage
90% 93%
Table 6.1. Results of the experiment, divided by the tasks.
6.5.1 Task completion and quiz assessment
Out of 41 tasks, there was only one task which was left incomplete. 37 tasks (90%) of
the tasks were completed with a full working implementation. 3 other tasks (7%) were
completed with a partial solution which did not completely meet the specification of
the task.
Task 2 (push pattern) turned out to be the most difficult task which took longest
to complete and involved most help from the moderator. At the end, one participant
left the task incomplete and two others could only achieve a partial solution. After the
first three sessions with participants, I asked them to skip from task one to task 3 and
return to task 2 after completing task 5, in order to avoid the participants being stuck
in a difficult task for a long time.
Similar results were achieved in the computational thinking quiz. At least 90% of
the answers were correct on each of two types of questions. This shows that the participants were able to recognize the computational patterns in the context of another
game to a high degree. This suggests that they may perform better in recognizing and
repeating the same patterns in a different game. However, as the focus of this experiment was on cooperation, I did not explore the transfer of computational thinking
patterns in further detail.
6.5.2 Cooperation
In practice, the game design session turned out to be quite similar to the situation of
a community in which a newcomer requires help from a superuser. On each task, the
participants continued on their own until they reached a point where they started to
seek help. I observed that they often seek help from the tutorials before asking for help
from the moderator. Table 6.1 shows that 57% of the participants sought help from
91
6.5 Results
the moderator to solve the tasks. In particular, in task 2, which was more difficult than
other tasks, all the participants sought help.
By going through the chat logs, I categorized the level of help they received to two
levels: minor help and major help. Though it was hard to draw a line between major
and minor help in some cases, I used the following questions to help me categorize the
level of help they received:
• Was the extent of communication between the participant and I limited to a quick
question with a quick answer, such as searching for the right condition or action
to use, or the participant asked further details towards the solution?
• Would the participant be able to complete the task on their own in a reasonable
amount of time without my help? If yes, the level of help is minor. Otherwise, it
is major.
I observed that the tutorials which were embedded into the programming environment were successful in helping novices figure out how a condition or action works.
Therefore, understanding the behavior of conditions and actions caused a little communication between the participants and the moderator. The sources of difficulties
were residing elsewhere. The following created most of the difficulties for participants
that caused them to seek help from the moderator:
• Language symbols: some of the participants misinterpreted the ‘See Myself’
condition and used it instead of ‘Stacked above’ condition in Task 3. However,
this was a simple issue which was fixed with a minor tip from the moderator.
• Order of execution: participants had difficulties understanding how the rules
were executed. Even though this is explained completely in the third part of
the programming tutorials which they watched prior to the game design session,
several participants needed at least a minor tip from the moderator to fix their
implementation.
• Interaction between agents: to implement the push pattern in Task 2, the
player has to interact with the ball to move the ball to the direction that it is
moving. The correct implementation uses the ‘make’ action in the player to call
a method in the ball. Participants faced two problems: 1) whether they have to
implement the solution in the player’s program, in the ball’s program, or in both.
2) how to use the make action properly. Using the make action is described in
detail in the second programming tutorial which they watched prior to the game
design session. However, they had difficulties understanding the concept of a
‘method call’. This caused Task 2 to become the most challenging task.
Each of the above sources of problems created several unique situations in which
participants sought help. The chat provided the most efficient channel to seek for a
92
6.5 Results
tailored help according to the situation. The synchronous aspect of the chat turned the
forum to be useless during the session.
Three participants (33%) reported that they have explored the games in showcase
at least once. Four participants (44%) reported that they have explored the games
created by others at least once. All 9 participants chose ‘definitely yes’ when they were
asked whether the video tutorials were helpful to solve the tasks. All of the 9 participants chose chat with the moderator as the most effective way to find a solution to
their tasks, when they were asked “Which way of collaboration was the most useful for
you in order to find the solution to the tasks? why do you think it was the most useful
one comparing to the others? " Here are some of the comments from participants,
comparing the chat with tutorials and exploring the other games:
• “Talking to the admin really helped. I got some questions that I had to answer in
order to get clues on how to do the programming. This really helped me in this
process rather than just watching the tutorials and trying to memorize them all."
• “one-to-one Chat. It was the most useful tool to find the solution because I received instant answers and from them I was able to realize where and how I can
find information or what I was doing right or wrong. In exploring the program of
games in showcase it was kind of difficult to understand where exactly I should
find the similar actions or conditions. "
6.5.3 Details of game design activities
The system successfully collected the details of the participants’ game design activities,
including tab switches between the worksheet and programming environment, and all
the programming activities as described in Section 6.3.1. The last four columns in
Table 6.1 provide information on the average performance of participants during each
task.
As I described before, I turned off the interactive programming mode of the environment in order to be able to precisely measure the participants’ game design flow,
i.e, whether they are programming or testing. The number of switches between the
worksheet and programming tabs shows that novices iterate quite a lot between the
programming and testing steps, even for simple tasks with one rule (Tasks 4 and 5).
This also shows the value of the interactive programming mode which facilitates the
iteration by removing any tab switches or even pressing a button to see the behavior
of agents at runtime.
The participants were very task-oriented, trying to get the tasks done. They limited
their communication with the moderator to finding a solution to their tasks. Therefore,
the number of chat messages which is shown in the last column of the table is fairly
close to messages that were used for finding a solution, although they include a few
messages that were communicated for the moderation of the session.
93
6.6 Conclusions
Interpreting the number of programming actions and chat messages require further
investigations. However, one interesting phenomenon is the close values of programming actions and chat messages for all the tasks excluding task 2, even though out
of these 33 task, only 15 asked for help (6 minor and 9 major). If we interpret each
programming action as an external ‘activity’ towards achieving a solution, and a chat
message as an internal ‘activity’ towards finding a solution, the measured numbers
suggest that novices employ the communication channel in parallel to and as much as
they carry out programming activities!
6.6 Conclusions
I conclude by a set of questions and answers that reflect on the research questions of
this chapter:
• Did novices rely on cooperation channels to solve their problems? This experiment confirms that novices comfortably used the provided cooperative features
of AgentWeb Website to seek help during solving computational thinking tasks.
In 57% percent of the tasks, participants sought help from the moderator. 3
out of 9 participants reported exploring showcase games and 4 out of 9 participants reported exploring the games created by other participants. The forum
was useless during the experiment due to the superiority of chat as a synchronous
channel to seek help.
• Did cooperation affect the success of novices in solving computational problems?
In 37% of tasks participants required major help from the moderator to accomplish the task. Based on my observations, in the case of a major help, the participant could not complete the task in a reasonable amount of time if she did not
receive a tailored guidance from the moderator. Therefore, cooperation would
lead them to a solution in a shorter amount of time.
• Did novices learn computational thinking concepts by using AgentWeb, i.e., a
cyberlearning platform rather than formal classroom education? Out of 82 computational thinking multiple-choice questions in the context of a different game,
75 questions (91%) were answered correctly.
This experiment paves the way to further experiments with more samples on each
individual tasks to better understand the effect on cooperation on the performance of
novices. During this experiment, I have been collecting information about the participants’ habits and opinions, in order to correlate them with their performance and use of
cooperation tools. For instance, I would like to compare the performance and outcome
of those who cooperated during solving computational problems with those who relied on their own capabilities to solve the problems. However, with the low number of
participants, conducting correlational studies between participants’ performance and
94
6.6 Conclusions
their game design behavior would not be statistically valid. I plan to continue this experiment with more participants to a point at which I can draw statistical conclusions
on participants’ game design behavior.
Chapter 7
Conclusions
In this thesis, I studied the feasibility of leveraging the World Wide Web as a cyberlearning platform for educational game design. This would allow for practicing and
gaining computational thinking skills in an open online platform benefiting from the
obvious advantages of Web applications. Such a platform has the potential to scale up
computer science education beyond the existing classroom settings by incorporating
computational thinking practices into a community of practitioners.
7.1 Research Contributions
This thesis verifies and validates the feasibility of using the open Web as a hospitable
platform for educational game design practices. The contributions of this thesis are:
1. Engineering an open-Web educational game design environment. I created
an educational game design environment, called AgentWeb, using only openWeb technologies, i.e., JavaScript, CSS, and HTML5, which enables users to
design, program and execute their games right from inside the Web browser.
AgentWeb is an Integrated Development Environment that consists of a visual
game scene editor, a visual programming language and its underlying compiler,
a game engine, a graphics engine, and an image editor. The engineering process included the requirements analysis, architectural design, implementation,
performance improvement and evaluation, and usability testing of AgentWeb.
AgentWeb’s game design model follows that of AgentSheets, a desktop-based educational game design environment which has been successfully used for teaching computational thinking in schools.
2. Building a cooperative game design Website. I designed and developed a
Website around AgentWeb which allows sharing and cooperation around the
game design activities. The Website encompasses the game design environment
and created games, and supports its users to explore each other’s games in an
95
96
7.2 Open Issues
open-source fashion. I created a set of sample games and a number of video
tutorials to facilitate the initial engagement for novices. Communication tools
such as chat, chatroom, and forum enable cooperation between the users.
3. Replicating educational practices in a cyberlearning platform. I conducted
an experiment in which a number of participants belonging to the Gen-Y population used AgentWeb to practice computational thinking activities. The results
show the successful use of the platform by novices for engaging in game design
activities and gaining computational thinking skills, and proves the critical role
of a cooperative Website, in particular, chat and instant access to explore the
inside of other games, in compensating for the lack of a formal teacher.
7.2 Open Issues
I developed AgentWeb as a proof-of-concept to verify the statements of this thesis.
However, numerous hours of using AgentWeb by novices has proven to me that as
a software, it goes beyond a simple proof-of-concept or prototype. I believe that by
addressing a number of issues, AgentWeb can be reliably employed by the public in
educational game design and computational thinking courses and activities. Regarding
the first statement of this thesis, the following is the list of open issues:
• In observing the use of the AgentWeb game design environment and the Website by novices, I have noticed a number of relatively easy-to-fix usability issues.
By addressing these issues, the user experience with the environment and the
Website will be improved.
• At the moment, AgentWeb is not compatible with Internet Explorer and Opera,
two major browsers that are used across the world.
• Delivering many JavaScript files to the client’s browser can be cumbersome. I
have developed a build system that merges and minifies AgentWeb’s code to just
a few files. However, several html widgets and snippets are loaded dynamically,
after the environment is loaded, including plugins of the visual language. This
can be further improved to increase the user’s experience in using the environment.
• Further educational materials such as showcase games and tutorials are needed
to cover the kinds of games that can be created with AgentWeb.
As for the second statement of this thesis, there is still a lot more to explore:
• In Chapter 4, I described a list of challenges that novices face in a cyberlearning
environment and suggested software tools and methods to compensate for the
lack of a teacher. The details of designing and integrating these tools into the
game design environment remained unexplored.
97
7.3 Future Directions
• In Chapter 5, I described the rationale behind the design of the Website to support cooperation between the members. The Website is fairly simple at this point,
with minimal possible scenarios for users to create a community around their
games. With today’s variety of possibilities for employing social media to the
benefit of cooperation and learning, by no means the developed Website is a
final solution.
• In Chapter 6, I conducted an experiment to verify the possibility of replicating
educational practices in a networked environment. Although the total number
of accomplished tasks by the participants is large enough to draw statistical conclusions to validate the second statement of this thesis, increasing the number
of participants is needed to allow me to analyze the outcomes per each computational thinking task and conduct correlational studies on participants’ performance and their use of the cooperation tools.
7.3 Future Directions
As an educational software, AgentWeb can be further enhanced in a number of ways
to increase the interest of learners and reach more novices:
• Enhancing the game design environment to support more rich content such as
sound and vector graphics will lead to more encouraging games for novices.
• Being built using open-Web technologies, AgentWeb runs on the major Web
browsers that support HTML5. This includes mobile platforms such as iOS
and Android, which are becoming popular for educational purposes. However,
AgentWeb’s IDE and its layout are not made for small screens and it does not support touch interactions. By designing a layout for mobile devices and supporting
touch interactions, AgentWeb can be used on mobile platforms.
To expand the use of the learning platform around AgentWeb, I consider a number
of options:
• Schools: AgentWeb can be employed in any classroom that is connected to the
Internet. AgentWeb users can access their personal workspace from any computer. This means that students can work on their games seamlessly, at home
and at school. The Website can be further extended to provide an interface
for teachers to set up an online classroom which is visible only to their students.
Each online classroom can have its own language, discussion board, assignments
and so on, similar to existing MOOC platforms such as Coursera1 .
1
https://www.coursera.org/
98
7.3 Future Directions
• Social Media: The Website can further employ social media to: 1) reach those
who are interested in gaining computational thinking skills but never had an
opportunity in the school. 2) allow a personalized experience for novices to share
and cooperate within their own online social network. This can be achieved by
integrating social networking platforms such as Facebook and Twitter into the
Website’s flow.
• Gamification: To lower the engagement barrier, and make it more fun to engage
in computational thinking activities, I would explore gamifying the game design
activities. For instance, users will collect points and badges for engaging in game
design activities. An advantage of AgentWeb over solitary game design environments is that group activities can be set up in which multiple users collaborate
or compete in game design activities, and a real-time scoreboard can provide
instant feedback on their collaboration/competition.
I hope that this thesis can pave the way to the building of a sustainable online
community in which computational thinking skills can be practiced from anywhere, at
anytime, and with minimum barrier for the learners.
Appendix A
AgentWeb’s Visual Programming
Language Specification
99
100
Data Type
Description
Text
Lets user input a text in a text box.
Key
Asks user to enter a key. Special keys such as arrow keys and space bar
are represented by their name.
Number
Lets user input a number in a text box. It checks the format automatically
and supports range checking and auto-correction of the entered value.
Relational Operator
A drop-down list allows the user to choose from (=,>=,<=,>,<,<>)
Stack Position
A drop-down list allows the user to choose from (above, immediately
above, below, immediately below, above or below)
Mathematical
Expression
A text box allows the user to enter a mathematical expression. The
expression must follow Visual AgentTalk Formula Syntax which is
described in: http://www.agentsheets.com/Documentation/windows/
Reference/tm_ti_formulasyntax.html
A mathematical expression parser translates the expression to its
equivalent JavaScript expression.
Direction
As shown in the picture, a visual interface allows the user to select one of
nine directions, i.e., one of the adjacent cells in the worksheet’s grid.
Agent List
A visual interface lets user pick up one of the agents. For each agent, it
shows only the first depiction, which is the main depiction of the agent.
Depiction List
Similar to the agent list, but it shows all the depictions of all the agents.
Worksheet List
A drop-down list allows the user to pick one of the worksheets which are
listed by their names.
Table A.1. Visual Data Types
101
Condition
Description
Arguments
Chance
[value]: Number
The rule will be executed with the chance
specified by its value. The value must be
between 0 and 100.
Current
worksheet
[worksheet name]: Worksheet List
If the running worksheet is the one chosen in
the worksheet list...
Empty
[direction]: Direction
If the cell at the given direction is empty...
Has
attribute
[direction]: Direction
[attribute]: Text
If a neighbor agent given in the direction has
a value for the specified attribute...
Is
[exp1]: Mathematical Expression
[op]: Relational Operator
[exp2]: Mathematical Expression
If (exp1 <op> exp2)
Key
[key]: Key
If the given key is pressed...
Next to
[op]: Relational Operator
[value]: Number
[depiction]: Depiction List
If the number of adjacent agents with the
given depiction satisfies the relation given by
op with the given value,...
Example: If there are 3 or more ‘X’ depictions
next to me .
Once
every
[time]: Number
Every once in a given time, this condition
holds. The time value is in seconds. The
smallest value is one millisecond.
See
[direction]: Direction
[depiction]: Depiction List
If the agent sees an agent with the given
depiction at the given direction...
See a
[direction]: Direction
[agent]:
Agent List
If the agent sees the given agent (regardless
of its depiction) at the given direction...
Stacked
[location]: Stack Position
[depiction]: Depiction List
If the agent is located at the given location of
an agent with the given depiction...
Stacked a
[location]: Stack Position
[agent]: Agent List
If the agent is located at the given location of
a given agent (regardless of its depiction)...
Table A.2. List of Conditions
102
Action
Description
Arguments
Broadcast
[agent]: Agent List
[method]: Text
Calls the given method of all the
instances of the given agent.
Change
[direction]: Direction
[depiction]: Image List
Changes the adjacent agent at the given
direction to the agent of given depiction
and sets its depiction accordingly.
Erase
[direction]: Direction
Erases the adjacent agent at the given
direction.
Invite Friends
(experimental)
none
If the player is logged in with her
Facebook account, the Facebook’s friend
invitation dialog opens up.
Make
[direction]: Direction
[method]: Text
Calls the given method of all an adjacent
agent located at the given direction.
Move
[direction]: Direction
Moves the agent one cell to the given
directions.
Move Random
[agent]:
Of all the adjacent agents with the given
direction, chooses one randomly and
moves on top of it.
New
[direction]: Direction
[depiction]: Depiction List
Creates a new agent instance of the given
depiction at the given direction and sets
its depiction accordingly.
Reset
Simulation
none
Resets the worksheet.
Set
[exp1]: Mathematical Expression
[exp2]: Mathematical Expression
Sets exp1 to exp2. exp1 is often an
attribute of the agent or a global variable.
Share on
Facebook
(experimental)
[caption]: Text
[description]: Text
If the player is logged in with her
Facebook account, the Facebook’s ‘Post
to the page...’ dialog opens up.
Show
Message
[message]: Text
Shows the given text in the status bar of
the worksheet.
Stop
Simulation
none
Stops the worksheet.
Switch
Worksheet
[worksheet name]: Worksheet List
Switches the running worksheet to the
given worksheet.
Transport
[direction]: Direction
Moves the agent and all the agents on top
of it one cell to the given directions.
Wait
[time]:Number
Pauses the execution for the given time.
Depiction List
Table A.3. List of Actions
Appendix B
Usability Evaluation Form
Actions/choices should be ranked according to following conventions: 0= nobody, 1=
somebody, 2 = more than half, 3= most.
B.1 Intellectual task
1. Description of user’s immediate goal:
2. How many users do you think will find it obvious that the action is appropriate to
the goal? 0 1 2 3
Why/Why not?
3. Executing the desired action: How many users do you think will find problems in
performing the action? 0 1 2 3
Why/Why not?
4. During the problem solving: How many users will find that obvious progress has
been made towards the desired goal? 0 1 2 3
Why/Why not?
5. Reaction to the satisfied goal: How many users do you think will recognise that
a task is completed? (to move to the game design environment to implement their
solution.) 0 1 2 3
Why/Why not?
6. Planning the achievement of a new goal: How many users do you think will find it
easy to satisfy a similar goal? 0 1 2 3
Why/Why not?
103
104
B.2 User Interface Task
7. How do you think this task could be supported better? i.e., By Improving the user
interface? Or by Providing Tutorials (text, video)? Please explain your ideas.
8. Assuming that the Website explains this task clearly using tutorials and videos, how
many users do you think will be able to learn by themselves how to accomplish this
task, e.g., at home, without being taught by an instructor? 0 1 2 3
Why/Why not?
B.2 User Interface Task
1. Description of user’s immediate goal:
2a. How many users do you think will find it obvious that the desired action is available? 0 1 2 3
Why/Why not?
2b. How many users do you think will find it obvious that the action is appropriate to
the goal? 0 1 2 3
Why/Why not?
3. Executing the desired action: How many users do you think will find problems in
performing the action? 0 1 2 3
Why/Why not?
4. Reaction to the satisfied goal: How many users do you think will recognise that a
task is completed? 0 1 2 3
Why/Why not?
5. Planning the achievement of a new goal: How many users do you think will find it
easy to satisfy a similar goal? 0 1 2 3
Why/Why not?
6. Assuming that the Website explains this task clearly using tutorials and videos, how
many users do you think will be able to learn by themselves how to accomplish this
task, e.g., at home, without being taught by an instructor? 0 1 2 3
Why/Why not?
7. How do you think this task could be supported better? i.e., By Improving the user
interface? Or by Providing Tutorials (text, video)? Please explain your ideas.
105
B.3 Computational Thinking Task
B.3 Computational Thinking Task
B.3.1 Matching with the sheet
1. Description of user’s immediate goal:
2. How many users do you think will find it obvious that the action is appropriate to
the goal? 0 1 2 3
Why/Why not?
3. Executing the desired action: How many users do you think will find problems in
performing the action? 0 1 2 3
Why/Why not?
4. During the problem solving: How many users will find that obvious progress has
been made towards the desired goal? 0 1 2 3
Why/Why not?
5. Reaction to the satisfied goal: How many users do you think will recognise that
a task is completed? (to move to the game design environment to implement their
solution.) 0 1 2 3
Why/Why not?
6. Planning the achievement of a new goal: How many users do you think will find it
easy to satisfy a similar goal? 0 1 2 3
Why/Why not?
7. How do you think this task could be supported better? i.e., By Improving the user
interface? Or by Providing Tutorials (text, video)? Please explain your ideas.
8. Assuming that the Website explains this task clearly using tutorials and videos, how
many users do you think will be able to learn by themselves how to accomplish this
task, e.g., at home, without being taught by an instructor? 0 1 2 3
Why/Why not?
B.3.2 Programming
1. Description of user’s immediate goal:
2a. How many users do you think will find it obvious that the desired action is available? 0 1 2 3
106
B.4 Computation Thinking Pattern Matching Sheet
Why/Why not?
2b. How many users do you think will find it obvious that the action is appropriate to
the goal? 0 1 2 3
Why/Why not?
3. Executing the desired action: How many users do you think will find problems in
performing the action? 0 1 2 3
Why/Why not?
4. Reaction to the satisfied goal: How many users do you think will recognize that a
task is completed? 0 1 2 3
Why/Why not?
5. Planning the achievement of a new goal: How many users do you think will find it
easy to satisfy a similar goal? 0 1 2 3
Why/Why not?
6. Assuming that the Website explains this task clearly using tutorials and videos, how
many users do you think will be able to learn by themselves how to accomplish this
task, e.g., at home, without being taught by an instructor? 0 1 2 3
Why/Why not?
7. How do you think this task could be supported better? i.e., By Improving the user
interface? Or by Providing Tutorials (text, video)? Please explain your ideas.
8. How many users do you think will find problems during writing the program with
or without performing Task 5, i.e., matching with the corresponding pattern ?
with matching to corresponding pattern: 0 1 2 3
without matching to corresponding pattern: 0 1 2 3
why/why not?
B.4 Computation Thinking Pattern Matching Sheet
107
B.4 Computation Thinking Pattern Matching Sheet
Action''
Keywords''
Behavior'Description'
Sample'Program'
Share'
'
Tell'friends'
'
When'something'
'
accomplished/happened'in'
the'game,'user'can'share'it'
on'Facebook.'
'
Example,'when'the'Golf'ball'
hit'the'hole,'share'with'
friends.'
Hit'
'
Collide'
'
Face'
'
Run'into'
Two'agents'collide'with'
'
each'other'
Ball’s'program'
'
Example:'when'Ball'hits'the'
ground,'bounce'back.'
Change'level'
'
Change'scene'
Go'to'another'scene'
'
Example:'in'the'Golf'game,'
when'the'ball'hits'the'hole,'
the'user'goes'to'the'next'
level.'
Create'
'
Generate'
'
Produce'
One'agent'creates'another'
agent.''
'
Example:''
shoot'action:'the'gun'
generates'the'bullet'when'x'
is'pressed.''
Change'
'
Look'like'
'
Become'
'
Alter'the'
figure'
'
Remove'
'
Absorb'
'
Erase'
'
Delete'
'
The'depiction'of'an'agent'
Ball’s'program'
changes'to'another'
depiction.'
'
Example:'When'the'ball'hits'
the'box'it'turns'red''
'
'
Gun’s'Program'
'
'
'
One'agent'erases'another'
agent'
'
Example:'snow'disappears'
when'hit'to'the'ground''
'
Snow’s'Program
'
Figure B.1. Computational Thinking Patterns, Page 1
108
B.4 Computation Thinking Pattern Matching Sheet
'
'
'
'
'
'
Push'
'
Press'
'
Poke'
One'agent'pushes'another'
Sokoban’s'program:'
agent'to'one'of'the'neighbor'
cells.'
'
Example:'
In'Sokoban'game:'the'
sokoban'pushes'the'box'(in'
the'picture'to'the'right).'The'
'
box'moves'if'the'neighbor'
cell'is'the'ground.'If'the'box' Box’s'Move_Right'method:'
moved,'it'will'tell'Sokoban'
to'move'right'as'well.'If'the'
box'can'not'move,'it'stays'
where'it'is'and'nothing'
happens.'
'
Sokoban’s'Move_Right'method:'
'
'
'
'
Keyboard'
control'
'
Key'press'
'
User'input'
'
'
'
The'agent'does'something'
when'a'key'is'pressed.'For'
instance,'in'the'provided'
example,'the'agents'moves'
left,'right,'up'and'down'
using'arrow'keys'
'
'
Figure B.2. Computational Thinking Patterns, Page 2
Appendix C
Programming Tutorial Manuscript
C.1 List of video tutorials
Name
Length
Description
Introduction to AgentWeb
3:58
Introduces the game design environment.
Creating a game
13:44
Create a game with AgentWeb, use agent
gallery, depiction editor, scene editor
Programming, Part 1
11:55
Programming environment, conditions and
actions, drag and drop, testing in worksheet,
Programming, Part 2
11:41
‘Forever’ method, other methods, calling
a method, ‘make’ action, managing the rule
Programming, Part 3
11:24
Order of execution
Programming, Part 4
10:54
Combining multiple conditions
Programming, Part 5
8:22
Combining multiple actions, tutorials inside the
environment, multiple depictions
Table C.1. List of video tutorials available on AgentWeb Website
C.2 Introducing AgentWeb
What is AgentWeb? (plot: showcases page) AgentWeb is a Web-based application
for creating 2D video games. You can create games such as PacMan, Frogger, Space
Invaders, Tetris, Sokoban and similar games with AgentWeb. Here is some examples
109
110
Name
How to check for two agents
C.2 Introducing AgentWeb
Length
Description
1.23
Stacked condition
How to ‘See’ adjacent agents
6:28
See condition, used with several actions
How to do ‘Time-based’ actions
3:58
Once-every condition
How to ‘Show a Message’
1:03
Show message action
1:04
New action
How to make an agent ’Move’
1:11
Move action
How to ‘Erase’ an agent
0:55
Erase action
2:42
Key condition, used with
residing in one cell
in the Status bar
how to create a new agent
in the scene
from the scene
How to use Keyboard to
control and move the agents
How to ‘Change’ the depiction
move action
2:26
Change action
1:44
Chance condition
of an agent
how to use a ‘Chance’ condition
Table C.2. List of video tutorials of conditions and actions that are embedded into
AgentWeb’s programming environment
of games created using AgentWeb. You can see Frogger, Sokoban and Space Invaders.
You can click on each game and play it, and explore the inside of the game to see how
the game has been created. [Play space invaders then move to frogger and show inside
of the frogger game].
How does it work? (plot: inside a frogger game) Of course you want to create
your own game. Maybe you have seen a game somewhere and you want to re-create
it or you have a story in your mind for a game. The way you use AgentWeb is that you
think about the story of the game you want to create and identify the game objects and
characters. For instance, if you want to create a frogger game, you have a frog, some
cars or trucks, some road, a river and so on. You create these objects, or as we call
them, agents, draw their depictions using an image editor, define their behavior using
a simple visual programming language, draw the agents on the game scene, and voila!
you made a game.
111
C.3 Creating a simple game with AgentWeb
Do I need to install any software? You can create a game and share it with others
right from the Website. So, there is no need to download and install any program.
As long as you have a Web browser such as Google Chrome or FireFox, you should be
set to use it right away. The current version does not work with Internet Explorer so
if you use Internet Explorer, you need to install Chrome or FireFox to use AgentWeb.
We recommend Chrome because it is really fast and is most suitable when it comes to
creating or playing games.
Do I need to know programming? Now you may think, "how can I do so, without
computer science or programming knowledge?". No worries! You can build games
quickly without having these skills. In fact, you will be doing some programming. But
we have made it fairly easy to do and provided several easy-to-learn tutorials to help
you learn how create your own game with AgentWeb. In the rest of this tutorial, we
are going to show you in details how to create a simple game and teach you what you
need to know from AgentWeb to quickly get your hands dirty with creating your game.
C.3 Creating a simple game with AgentWeb
Creating a new game: Now let’s see in details how to create a game with AgentWeb.
Let’s start a new game, called ‘the ball’. We can’t run it yet as we have not created
anything. So, we go to the design page. Here we see the game design environment
that lets us create the game and save it. First it asks for the size of the game objects,
or as I mentioned before, we call them ‘agents’. Here you see the default is 32 by 32
pixels, which is a good size to start with. Let it be as it is and press the create button.
Let’s start by creating a ball. I go to the ‘Agent Gallery’ and press this ‘add’ button. I
call our agent ‘ball’. Here we go. It created a ball agent and added a default depiction
to it. You can edit the depiction by double-clicking on it, which switches to the editor
tab in here, and loads the depiction so you can edit it. Also, when you click on the
agent or depiction, it selects the agent.
C.4 Using the Depiction Editor
Items to demonstrate:
• clean the canvas
• choose the drawing tool
• erase
• change the fill/stroke color
• fill in a closed area
112
C.5 Drawing the game scene
• save the depiction
• undo
• zoom and resize
C.5 Drawing the game scene
Items to demonstrate:
• Drawing tools
• Move an agent
• Play, stop, and step actions
• Save, reset and clear worksheet
• Change the size of worksheet
• Add new worksheet
C.6 Programming
Each agent has its own program: Now let’s move the programming part. By programming, you define the behavior of the agent, such as how it moves or reacts to
other agents during the game. The first thing you need to know, which is important,
is that each agent has its own program. When you choose an agent from the agent
gallery, you can see its program in the program tab. So, if you choose the ball, you see
the ball’s program, and if you choose the ground, you see the ground’s program. Now
let’s go back to the ball’s program.
Three panes, composing program from conditions and actions: Here you see
that the program tab has three parts: Conditions pane on the left, Program pane in
the middle, and Actions pane on the right side. The conditions and actions panes are
exactly the same for all the agents. What changes is the program pane. What you do
is to "compose" the program by dragging and dropping the conditions and actions you
need, to define the behavior of the agent.
Compose using drag-n-drop, using actions: Let’s write a simple program first
and then I explain it in details. I want that the ball moves continuously to the right
of the scene. So, I go to the program of the ball, not the ground, and in this method
called ‘Forever’, that I explain more about it later, we take this action, called move,
and drag in into this rule, on the action part, you see the drag sign changes to green
which means that the item can be dropped, otherwise it does not allow dropping, so if
113
C.6 Programming
I take an action and move it to the condition part of a rule or anywhere else and drop
it, nothing happens. Then I set the direction of the move to right.
Run the worksheet to see the behavior: Now let’s go to the game scene, and
run the game to see the behavior of the agent. First we go to the worksheet pane. at
the bottom there are the buttons on the left to run or stop the game scene. I press
this green play-like button to run the scene. As soon as I run the scene, the ball agent
moves to the right quickly. This is because the Forever method is executed repeatedly,
as fast as my computer can, and each time runs this move action to move to the right.
Agent program and instances (inheritance): One thing you should note is that
the program that we wrote defines the behavior of all the instances of that agent. In
our case, the move-to-right program is for the ball agent. Therefore, all the balls on
the worksheet have the same behavior of moving up quickly. We can see this by simply
adding other balls on the game scene. As soon as we add them, we see this behavior
of moving to right for the newly added ball.
Using conditions: Now to add more control on how the agent behaves you can
add some conditions. for instance we can use a once-every condition to tell the agent
to move one step every once in a while, and you can determine the interval of the
move in here. For example, we set it to move to the right every 1 second and let’s try
what happens. Now we have a moving ball and we control its moving speed. So far,
so good.
Now let’s get a little more into the details of the programming environment. One
thing you have noted is that we are adding conditions and actions inside a rule of a
method called forever. So, there are two things you should know about: (i) what a
method is; (ii) what a rule is.
Methods: Methods are pieces of programs that define the behavior of agents.When
an agent has a behavior that needs to happen in different situations, instead of programming that behavior several times, we create a method for that behavior and execute it. For example, if the ball bounces both when it hits the wall and when the
player presses the ‘Space’ button, instead of programming the bounce behavior twice,
you compose it once in a separate method and execute that method whether when the
user presses the ‘Space’ button or the ball hits the wall. Each method consists of one
or more rules. Each rule consists of conditions and actions. You can create an empty
method by pressing this ‘Add Method’ button and choosing a name for it, or choose a
method and press the ‘duplicate’ button which creates a copy of the method with the
same rules, conditions and actions. Note that methods should have different names,
i.e., no two methods can have the same name in one agent program.
Forever method: There are two types of methods. One special method is called
‘Forever’, which is executed all the time when the game is running. It means that
whatever program you write in the forever, it keeps repeating. For example, we saw
that if we use a move action, it keeps repeated. You can have only one ‘Forever’ method
for each agent. As a matter of fact, the system does not allow you to have two methods
114
C.6 Programming
called ‘Forever’.
Other methods: As I said, you can create other methods either using add method
button or duplicate button. Now here I create a method called move-left, which is
essentially similar to the move-right program we have written, except that the agent
moves right this time. So, we can either create a new method and add the once-every
condition and move action, or we can duplicate the forever method, and make the
changes we want in it, change its name to move-left and change the direction of the
move to the left. The system does not care how you compose the program. It only
executes the program you have written.
Executing a method: Methods are not automatically executed during the game.
Rather, you should explicitly execute them, or as we say in programming ‘call the
method’. You can call a method whether from inside the same agent or from an adjacent agent. For both, you will have to use the the ‘make’ action. Enter the name of
the method that you want to execute inside the make action. If the method you are
calling belongs to the same agent, leave the direction as it is and do not change it.
If you are calling the method of an adjacent agent, choose the direction here in the
make action. The way it works is that you put the make action in a rule which when
it is executed, it causes the method to be executed as well. For instance, here I call
the move-left method from inside the Forever method. Because the Forever method
is executed repeatedly, whenever the rule with the make action is executed, the make
action calls the move-left method, which causes the move-left method to be executed.
Managing the rules: Now that you know how to deal with methods, let’s look at
how to use the rules. Each method can have several rules. When you create a method,
it already has an empty rule for you. A rule is a placeholder for condition and action.
You can add several conditions and actions to each rule. There are a couple of way to
create a new rule. First, you can use the ‘New Rule’ button to create a new rule, but
you should choose a method first, so that the system knows to which method it should
add the rule. You can reorder the rules by dragging them around. You can even move
the rules between the methods. You can select a rule, and press the duplicate button
to copy it. Or you can drag a rule and press the control button or command button on
the Mac to copy a rule.
The order of execution: An important thing to know how the rules inside a
method are executed. The first thing to remember is that when a rule does not have
any conditions, it means that the rule holds and will be always executed. Here we
add a rule for the ball to move right with no condition and as we saw the ball keeps
moving to the right all the time because the forever method is executed repeatedly and
because the rule does not have any condition.
Now you may wonder what happens if we have two or more rules? Which one is
executed? Here you need to have a very basic knowledge of logic. When a method is
executed, the rules get what we call tested, one by one, in order from top to bottom. Testing
means that all the conditions of a rule are evaluated. Evaluation means whether that
115
C.6 Programming
condition holds or not. If all the conditions of a rule hold, all of its actions will be
executed and the method will be terminated, which means that the other rules will
not be tested or executed. To see this in practice, if I add a rule to move left, with no
condition, and put it after the first rule, which moves the ball to the right, we see that
the second rule does not happen. If I move this rule on the top of other one, then the
ball moves left, rather than right. It shows that the order of testing the rules is from
top to bottom and whenever one rule is executed, other ones are skipped.
To elaborate this with a more comprehensive example, here I create three rules
with three types of conditions: the first rule is to move left with no condition. The
second is to move up when the ‘W’ key is pressed. Now if the move up rule is after the
move left rule, it never works as we saw before, no matter how many times you press
the ‘W’ key, because the move left rule is always running. But if you move the move
up rule to before the move left rule, it starts working because now it is tested before
the move left rule. Notice that the move left rule is still working because there are a
lot of times when you do not press the ‘W’ button and therefore the move top rule is
not satisfied which causes the system to test the second rule, which is move left with
no condition, and it is always executed. The move left is not executed whenever you
press the ‘W’ key because in that case only move up is executed and move left rule is
disregarded. Now what happens if we add a rule after the second rule? say a move
down rule. You guessed right! It is not executed.
Combining multiple conditions: Now what happens if we use more than one
condition in a single rule? In fact, there are cases in which combining the conditions
are useful. I explain some of them here:
• once-every AND see: e.g., in a frogger game the car needs to move only on the
road once in a while.
• once-every AND chance: e.g., in a frogger game the tunnel generates a car on
the right once in a while and with some percent of chance, to avoid the ordered
generation of the cars.
• once-every AND see AND chance: e.g., the car moves randomly once in a while
only on the road.
• key AND see: e.g., the ball moves around with arrow keys only on the ground,
i.e., it won’t move outside of the field.
One thing you should notice is that the rule is satisfied only if all of its conditions
are satisfied. If only one of the conditions does not hold, the rule is not satisfied and
therefore none of the actions of that rule will be executed and the system continues to
the next rule of the method. This way, you can prioritize which rule should be executed
first, as if that rule is executed, other rules will be disregarded.
Combining multiple actions: Similar to using multiple conditions in the same
rule, you can use multiple actions in the same rule. When the conditions of a rule
116
C.6 Programming
hold, all of its actions are executed from the top to the bottom. A simple example for
this is to move left and show a message. You can combine the actions the way you
want and in the order you want.
Other programming conditions and actions, video tutorials: To find out more
about each condition and action, you can move your mouse on the name of the condition or action and press this green play button to watch a tutorial that shows exactly
how that condition or action works.
Agents can have multiple depictions: One last thing to mention is that each
agent can have several depictions. You can add a depiction to an existing agent from
the agent gallery. Under the main depiction of each agent there is plus button which
you press and enter the name of the new depiction. The system adds the new depiction
under the agent and you edit it as we seen before.
Agent depictions have the same program: Note that all different depictions of
one agent have the same program, which means that they behave the same in the
game scene. For example, in a car game you can have a car agent with two different
depictions. Both of them do the same thing in the scene, they move from one side of
the road to another side of the road. Their different depictions are only for adding
more variety to the game.
Appendix D
Evaluation Forms
117
118
D. Evaluation Forms
D.1 Registration Form
D.1.1 Computer Knowledge
1) What is your skill level at using computers?
( ) Non-existent
( ) Poor
( ) Fair good
( ) Very good
( ) Excellent
2) In average, how often do you use a Web browser for ANY Web applications (including Google,
Facebook, Youtube, Yahoo, News Websites, Wikipedia, Amazon, Online Games, etc.)?
( ) Not at all (zero hours a day)
( ) Very little (30 minutes a day)
( ) Regularly (1 hour a day)
( ) Quite often (2 hours a day)
( ) Very often (3 to 4 hours a day)
( ) More than 4 hours a day
3) Have you ever taken a computer science/Informatics/Programming course?
( ) Yes
( ) No
4) What have you learned from the class?
5) Have you ever written a computer program?
( ) Yes
( ) No
( ) I don't know
6) Name some of the programming languages that you have used:
7) Have you ever made your own web page?
( ) Yes
( ) No
( ) I don't know
8) What programs/languages have you used to create your webpage?
D.1.2 Participant's information
119
D. Evaluation Forms
9) Gender
( ) Male
( ) Female
10) Age
____________________________________________
11) Country
____________________________________________
12) State
____________________________________________
13) Profession
____________________________________________
14) Education
( ) Elementary School
( ) Middle School
( ) High School
( ) Graduated high school or equivalent
( ) Some college, no degree
( ) Associate degree
( ) Bachelor's degree
( ) Post-graduate degree
15) In which country did you go to school?
____________________________________________
16) What is your knowledge of English language?
( ) Poor or non-existent
( ) Basic
( ) Good
( ) Very Good
( ) Native Speaker
120
D. Evaluation Forms
D.2 Pre-test questionnaire
D.2.1 Computer Usage
1) Name:
____________________________________________
2) If you are a student, what is your major?
____________________________________________
3) If you work, what is your field(s) of experience?
____________________________________________
4) During a week, how many hours do you use a computer for any purpose?
0 1-5 6-10 11-15 16-20 21-25 26-30
More than
30
Hours spent using a desktop or laptop
computer (Mac,PC,Linux,etc)
() ()
()
()
()
()
()
()
Hours spent using a smart phone (iPhone,
Android, etc) except talking on the phone
() ()
()
()
()
()
()
()
Hours spent using a Tablet (iPad, Galaxy Tab, ( ) ( )
etc)
()
()
()
()
()
()
Hours spent using an ebook reader (Kindle,
Nook, etc)
()
()
()
()
()
()
() ()
5) How often do you use programs such as Google, Facebook, Wikipedia, Online Games (internet
programs in general)?
( ) Not at all
( ) Very little (less than two hours a week)
( ) Regularly (2-5 hours a week)
( ) Quite often (5-10 hours a week)
( ) Very often (10-20 hours a week)
( ) More than 20 hours a week
6) Name some of the internet/Web programs that you use regularly.
____________________________________________
121
D. Evaluation Forms
7) Check any of the applications that you use or have you used for communication:
[ ] Skype
[ ] Google Talk
[ ] Google Hangout
[ ] iChat
[ ] Windows Live Messenger
[ ] Yahoo Messenger
[ ] Facebook chat
[ ] AIM
[ ] Any other Instant Messaging applications that you use:
D.2.2 Gaming Experience
8) How often do you play computer games?
( ) Not at all
( ) Very little (less than two hours a week)
( ) Regularly (2-5 hours a week)
( ) Quite often (5-10 hours a week)
( ) Very often (10-20 hours a week)
( ) More than 20 hours a week
9) Please rate how much and where do you use computer games.
On a Game Console (XBOX, Wii,
PlayStation, Nintendo DS,...)
On a Smartphone (iPhone, Android,
Windows Mobile, ...)
On a computer but not online
()
()
()
Half of a
time
()
()
Most of
the time
()
()
()
()
()
()
()
()
()
()
()
()
()
On internet, online
()
()
()
()
()
()
Not at all Very little Sometimes
10) Name some of your favorite games:
11) Please rate what kind of computer games you play.
Often
122
D. Evaluation Forms
Massive Multi-player games or Virtual
Worlds such as Second Life, World of
Warcraft, or similar ones.
I Dont
Not at Very Somet Half of
Most of
know this
Often
A lot
all
little imes a time
the time
kind of
game
()
()
()
()
()
()
()
()
Casual games such as Angry Birds,
Solitaire, or similar ones
()
()
()
()
()
()
()
()
Arcade games: Frogger, Pac Man, Space
Invaders
()
()
()
()
()
()
()
()
First Person Shooter Games such as
Counter Strike or similar ones
()
()
()
()
()
()
()
()
Platformers such as Little Big Planet or
similar ones
()
()
()
()
()
()
()
()
12) If you play other genres of computer games that are not listed above, please mention them in here:
____________________________________________
13) Have you ever created/designed a computer game?
( ) Yes
( ) No
14) If so, please name them and explain how did you create/design them:
____________________________________________
D.2.3 Social Networking experience
15) Do you use social networking programs? (please select those that apply)
[ ] Facebook
[ ] Twitter
[ ] Google+
[ ] LinkedIn
[ ] Youtube
[ ] Other social networks (e.g., Orkut, Badoo, Bebo, Tagged, NetLog, Friendster, ...):
16) How much do you use social networking programs?
( ) Not at all
( ) Very little (less than two hours a week)
123
D. Evaluation Forms
( ) Regularly (2-5 hours a week)
( ) Quite often (5-10 hours a week)
( ) Very often (10-20 hours a week)
( ) More than 20 hours a week
17) Task-oriented personalities tend to:
- Focus on their to-do list and the things they hope to accomplish.
- Be concerned with productivity and efficiency.
- Have concrete goals and detailed lists.
People-oriented personalities tend to:
- Focus on the needs of the people around them.
- Be concerned with building relationships and keeping people happy.
- Place more importance on the feelings and happiness of people than on their to-do list.
Do you consider yourself as Task-oriented or People-oriented?
( ) Task-oriented
( ) People-oriented
( ) I don't konw
18) Do you consider yourself as introvert (reserved, introspective) or extrovert (outgoing, sociable)?
( ) Introvert
( ) Extrovert
( ) I don't know
19) Please rate yourself according to following questions:
How would you rate your personality as being
"sociable" in your day-to-day life?
How would you rate your online "social
presence"? (such as sharing, blogging, tweeting,
commenting on blog posts, chatting, updating
your profile picture, changing your status,...)
Very low
()
Low
()
Average
()
High
()
Very high
()
()
()
()
()
()
20) How often do you share something (photos, videos, links, news, etc) with your friends on your
online social network (Facebook, Twitter, Google+, etc)? Please choose the closest answer.
( ) Never
( ) 1 or 2 times a month
( ) 1 or 2 times a week
( ) 1 or 2 times a day
( ) 3-5 times a day
( ) more than 5 times a day
124
D. Evaluation Forms
D.2.4 Computer Knowledge
21) When something goes wrong with your computer, how frequently do you ask friends or family
members for help fixing it?
( ) Very infrequently
( ) Somewhat infrequently
( ) Neither frequently nor infrequently
( ) Somewhat Frequently
( ) Very frequently
22) When you want to install a new computer program, how frequently do you ask friends or family
members to help you install it?
( ) Very infrequently
( ) Somewhat infrequently
( ) Neither frequently nor infrequently
( ) Somewhat Frequently
( ) Very frequently
23) Please rate the following sentences.
Installing a new program is easy.
Strongly
Disagree
()
()
Neither agree
nor disagree
()
Disagree
()
Strongly
Agree
()
Agree
Installing a new program is a waste of time.
()
()
()
()
()
I love the installation part of programs.
()
()
()
()
()
Installing a new program is a burden for me
to use or try it.
Installing a new program is boring.
()
()
()
()
()
()
()
()
()
()
I prefer to use Web-based programs
because they do not need installation.
I prefer to use Web-based programs
because they keep and manage their data
(photos, text, videos, project files, etc.)
I prefer to use Web-based programs
because I can access my data/files from any
computer.
I do not like Web-based programs because
they often require registration (i.e., ask for
a user name and email address) on the
Website.
I prefer to use Web-based programs
because I can access my account and data
from any computer.
I prefer Web-based applications to desktopbased applications.
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
125
(leave this empty if you do not have a
facebook account) I feel comfortable to use
my Facebook account to register/login on
other Websites (using a blue 'Facebook
Login' button)
I prefer to download files (texts, images,
videos, etc) first and then read/watch them
outside of the Web browser rather than
reading/watching them right away inside
the Web browser.
Desktop-based applications have better
graphics comparing to Web-based
applications
Desktop-based applications are faster than
their Web-based counterparts.
I prefer desktop-based applications to Webbased applications.
D. Evaluation Forms
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
D.2.5 Computer Science Knowledge
24) Would you be interested in taking an introductory computer science course?
( ) Definitely not
( ) Probably not
( ) Maybe yes, Maybe no
( ) Probably yes
( ) Definitely yes
25) Do you think if you learn the basics of computer science, it will help you decide better when solving
day-to-day life problems or improve your career opportunities?
( ) Definitely not
( ) Probably not
( ) Maybe yes, Maybe no
( ) Probably yes
( ) Definitely yes
26) Do you think learning computer science/programming concepts is a solitary activity or a
collaborative activity?
( ) Completely solitary
( ) Mostly solitary
( ) Partly solitary, partly collaborative
( ) Mostly collaborative
( ) Completely collaborative
126
D. Evaluation Forms
27) Please rate the following sentences.
I am interested in an introductory computer science
course in which I learn how to program a textual
programming language such as C/C++/Java/
JavaScript/Python/etc.
I am interested in an introductory computer science
course in which I learn how to create applications for
smart phones such as iPhone/Android/Windows
Mobile.
I am interested in a video game design course in
which I learn the basic concepts of computer science
and programming while creating computer games.
I am interested in a Web design course in which I
learn how to create Web pages/sites using HTML/
CSS/JavaScript.
Definitely Probably
Probably Definitely
Neutral
No
No
Yes
Yes
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
28) Do you think you could learn an advanced computer language like Java or C++?
( ) Definitely not
( ) Probably not
( ) Maybe yes, Maybe no
( ) Probably yes
( ) Definitely yes
127
D. Evaluation Forms
D.3 Task Completion
1) Task Number
( ) 1 (move the PLAYER around with arrow keys)
( ) 2 (program the PLAYER to move the ball )
( ) 3 (When the BALL reaches on the top of the GOAL, the status bar shows a 'GOAL' message)
( ) 4 (When the key 'X' is pressed, the CENTER creates a new BALL on its right)
( ) 5 (When the GOALIE sees a BALL on its left, the GOALIE removes the BALL)
2) Did you attempt to complete this task?
( ) Yes
( ) No
3) Did you manage to complete this task?
( ) Yes
( ) No
4) Rate the following about this task
Very low
()
Low
()
Neutral
()
High
()
Very high
()
Technicality
()
()
()
()
()
It was fun to complete this task
()
()
()
()
()
It was boring to complete this task
()
()
()
()
()
It was nerve-racking to complete this task
()
()
()
()
()
It was joyful to complete this task
()
()
()
()
()
Difficulty
5) What sources did you use as help to find a solution for this task? check all that apply.
[ ] Tutorials
[ ] Users (including admin) on the Website through one-to-one Chat
[ ] Users (including admin) on the Website through Chatroom
[ ] Users (including admin) on the Website through Forum
[ ] Program of the showcase games
[ ] Program of the games created by other users
[ ] Other ways (please explain in details)
6) Do you think you could have completed this task without the help of other participants (including
admin)?
( ) Yes
( ) No
( ) Maybe yes, maybe no
128
D. Evaluation Forms
D.4 Computational Thinking Quiz
1) Name
____________________________________________
This is a Torch:
!
You are making a computer game for olympic games. The following shows part of the program of an agent
for an olympic runner, doing SOMETHING with the torch.
2) What is he doing?
( ) when the olympic runner moves to the right, it creates a torch.
( ) every 5 seconds, the runner creates a torch.
( ) When the key X is pressed, the olympic runner creates a torch on his right.
( ) the olympic runner moves to the right and creates a torch.
3) Choose one of the following behaviors that best describes the program in the previous.
( ) Absorption/Deletion
( ) Generation/Creation
( ) Collision
( ) Push
( ) Pull
( ) Diffusion
129
This is a Ground agent:
D. Evaluation Forms
!
The following is the program of a Ball agent.
!
4) What does the above program do?
( ) when the ball hits the ground, it bounces back up.
( ) The ball bounces back up, with 50 percent of chance.
( ) The ball moves down until it reaches the ground and then stops.
( ) When the ball hits the ground, it is removed from the game scene.
5) Choose one of the following behaviors that best describes the program in the previous.
( ) Absorption / Deletion
( ) Generation / Creation
( ) Collision
( ) Keyboard control / Cursor control
( ) Push
( ) Pull
( ) Diffusion
130
This is a Ground agent:
D. Evaluation Forms
!
You are writing the program for a falling snow agent. Following is the program of the falling snow. !
6) What does the above program do?
( ) when the snow reaches the ground, it stops on top of the other snow that is already on the ground.
( ) as soon as the snow reaches the ground, it removes itself from the scene.
( ) when the ground moves down, it sees the snow and removes the snow.
( ) when the ground sees the snow, it collects the snow on top of itself.
7) Choose one of the following behaviors that best describes the program in the previous.
( ) Absorption / Deletion
( ) Generation / Creation
( ) Collision
( ) Keyboard control / Cursor control
( ) Push
( ) Pull
( ) Diffusion
131
D. Evaluation Forms
You are writing the program for a car agent.
Note: on the keyboard, arrow keys are: UP arrow, DOWN arrow, LEFT arrow, and RIGHT arrow.
!
8) What does the above program do?
( ) It lets the player to move the car to up, down, left and right, by pressing W, D, S, and A on the keyboard.
( ) It lets the player to turn the car by pressing all the arrow keys together at the same time.
( ) It shoots an arrow out of the car in all directions, when the player presses arrow keys.
( ) It lets the player to move the car to up, down, left and right, by pressing the arrow keys (up, down, left
and right) respectively.
9) Choose one of the following behaviors that best describes the program in the previous.
( ) Absorption / Deletion
( ) Generation / Creation
( ) Collision
( ) Keyboard control / Cursor control
( ) Push
( ) Pull
( ) Diffusion
132
This is a BOX agent.
D. Evaluation Forms
!
The BOX has a Move-Right method as shown here: !
There is a MAN agent with the following program:
!
10) What does the program of the MAN agent do?
( ) When the right arrow key is pressed, the MAN moves to the right and also moves the BOX to the right.
( ) When the MAN sees a BOX to the right, the MAN moves to the right and also moves the BOX to the
right.
( ) When the right arrow key is pressed, only if there is a BOX at the right side of the MAN, then the MAN
moves to right and also moves the BOX to the right
( ) When the right arrow key is pressed, only if there is a BOX at the right side of the MAN, then the MAN
moves to the right on top of the BOX.
11) Choose one of the following behaviors that best describes the program in the previous.
( ) Absorption / Deletion
( ) Generation / Creation
( ) Collision
( ) Push
( ) Pull
( ) Diffusion
133
D. Evaluation Forms
D.5 Qualitative Questionnaire
1) In order to find the solution to the given tasks, did you use any of the collaboration tools (chat,
chatroom, forum, exploring the program of games in showcase, exploring the program of games
shared by the other participants) ?
[ ] Yes
[ ] No
2) How did you collaborate with the others to solve the tasks? provide some examples of your
collaboration if you can.
3) Which collaboration tools did you use at least once? check all that apply.
[ ] one-to-one Chat
[ ] Chatroom
[ ] Forum
[ ] Exploring the program of games in showcase
[ ] Exploring the program of the games shared by other participants
[ ] Other ways you collaborated with others (please explain)
4) Which way of collaboration was the most useful for you in order to find the solution to the tasks?
why do you think it was the most useful one comparing to the others? please explain.
5) Could you find the solution to the tasks without provided collaboration tools (one-to-one chat,
chatroom, forum) ? for instance, if you were to use a solitary desktop-based game design environment
with no support for collaboration.
( ) Yes
( ) No
( ) Maybe yes, Maybe no
( ) I would able to solve the problems by myself if ...: _________________
6) You were given 5 tasks to complete. Was there any task left that you could not solve ?
( ) Yes
( ) No
7) Which tasks could not you solve? what was the reason?
8) For those tasks which you did not solve, did you try to find the solution by asking from others
(including admin)? did you receive any support from the other participants (including admin)? please
explain why others (including admin) could not or did not help you to find the solution.
134
D. Evaluation Forms
9) Do you have any further comments or suggestions for us regarding the game design environment,
the Website, or the collaboration tools?
10) Was the collaboration with others (including admin) helpful to solve the tasks?
( ) Definitely No
( ) Maybe No
( ) Maybe Yes, maybe No
( ) Maybe Yes
( ) Definitely Yes
D.5.1 Tutorials
11) There were 5 Programming Tutorials (AgentWeb Tutorials - Programming, Part 1-5). Which ones
have you wateched?
[ ] Part 1
[ ] Part 2
[ ] Part 3
[ ] Part 4
[ ] Part 5
12) Were the programming tutorials helpful to solve the tasks?
( ) Definitely No
( ) Maybe No
( ) Maybe Yes, maybe No
( ) Maybe Yes
( ) Definitely Yes
13) There were some video tutorials inside the Program tab, for each condition and action, that you
could watch by clicking on the green play button. Did you watch any of them? Which ones did you
watch?
14) Were they helpful to solve the tasks? (the tutorials inside the Program Tab)
( ) Definitely No
( ) Maybe No
( ) Maybe Yes, maybe No
( ) Maybe Yes
( ) Definitely Yes
135
D. Evaluation Forms
15) The video tutorials were:
Neither
Strongly
agree
Strongly
Disagree
Agree
Disagree
nor
Agree
disagree
()
()
()
()
()
Fun
Boring
()
()
()
()
()
Confusing
()
()
()
()
()
Difficult
()
()
()
()
()
Annoying
()
()
()
()
()
Frustrating
()
()
()
()
()
Easy to
learn
()
()
()
()
()
16) Did you face any problems with video tutorials that made them less interesting to follow or difficult
to understand? such as the voice of the speaker, the quality of the video, or it was too long, or too
short, or any other problem.
17) Do you have any suggestions/comments to improve the tutorials?
D.5.2 Game Design Session
18) Today's game design session was:
Neither
Strongly
agree
Strongly
Disagree
Agree
Disagree
nor
Agree
disagree
()
()
()
()
()
Fun
Boring
()
()
()
()
()
Confusing
()
()
()
()
()
Difficult
()
()
()
()
()
Annoying
()
()
()
()
()
Frustrating
()
()
()
()
()
Easy to
learn
()
()
()
()
()
19) Will you consider participating in a similar game design session?
[ ] Yes
[ ] No
[ ] I don't know
20) May we keep your email and contact you for future tests/evaluations?
[ ] Yes
[ ] No
136
Bibliography
[1] AAUW Educational Foundation Commission on Technology, Gender, and
Teacher Education. Tech-Savvy: Educating Girls in the New Computer Age.
Technical report, 2000.
[2] N. Ahmadi, M. Jazayeri, and M. Landoni. Enabling Cooperative Educational
Game Design on the Web. Cooperative Design, Visualization, and Engineering,
pages 57–64, 2012.
[3] N. Ahmadi, M. Jazayeri, and M. Landoni. Helping Novice Programmers to Bootstrap in the Cloud: Incorporating Support for Computational Thinking into the
Game Design Process. Advanced Learning Technologies (ICALT), 2012 IEEE 12th
International Conference on, pages 349–353, 2012.
[4] N. Ahmadi, M. Jazayeri, F. Lelli, and S. Nesic. A Survey of Social Software
Engineering. 1st International Workshop on Social Software Engineering and
Applications, pages 1–12, Aug. 2008.
[5] N. Ahmadi, M. Jazayeri, and A. Repenning. Engineering an Open-Web Educational Game Design Environment. The 19th Asia-Pacific Software Engineering
Conference, page 10, Oct. 2012.
[6] N. Ahmadi, F. Lelli, and M. Jazayeri. Supporting Domain-Specific Programming
in Web 2.0: A Case Study of Smart Devices. 21st Australian Software Engineering
Conference (ASWEC), pages 215–223, 2010.
[7] N. Ahmadi, A. Repenning, and A. Ioannidou. Collaborative end-user development on handheld devices. IEEE Symposium on Visual Languages and HumanCentric Computing(VL/HCC), pages 237–241, 2008.
[8] B. Alexander. Web 2.0: A new wave of innovation for teaching and learning?
EDUCAUSE review, 41(2):32–44, 2006.
[9] J. Allaire. Macromedia Flash MX—A next-generation rich client. White paper,
Macromedia, page 10, 2002.
137
138
BIBLIOGRAPHY
[10] K. M. Anderson. Supporting software engineering with open hypermedia. ACM
Computing Surveys, 31(4es), Dec. 1999.
[11] S. M. Barnett and S. J. Ceci. When and where do we apply what we learn?: A
taxonomy for far transfer. Psychological bulletin, 128(4):612–637, 2002.
[12] A. R. Basawapatna, K. H. Koh, and A. Repenning. Using scalable game design to
teach computer science from middle school to graduate school. In Proceedings of
the fifteenth annual conference on Innovation and technology in computer science
education, pages 224–228, New York, NY, USA, 2010. ACM.
[13] G. Bekebrede, H. J. G. Warmelink, and I. S. Mayer. Reviewing the need for gaming in education to accommodate the net generation. Computers & Education,
57(2):1521–1529, Sept. 2011.
[14] M. Ben-Ari. Constructivism in computer science education. In SIGCSE ’98:
Proceedings of the twenty-ninth SIGCSE technical symposium on Computer science
education, 1998.
[15] T. Berners-Lee. Weaving the Web: The Original Design and Ultimate Destiny of
the World Wide Web. HarperBusiness, 1 edition, Nov. 2000.
[16] T. Berners-Lee, R. Cailliau, and B. Pollermann. World-Wide Web: The Information Universe. Communications of the ACM, 37:76–82, Feb. 1992.
[17] C. Boldyreff, D. Nutter, S. Rank, M. Smith, P. Wilcox, R. Dewar, and D. Weiss.
Environments to Support Collaborative Software Engineering. In 2nd Workshop
on Cooperative Supports for Distributed Software Engineering Processes, pages
25–28, Oct. 2004.
[18] G. Booch and A. Brown. Collaborative Development Environments. ADVANCES
IN COMPUTERS, 59:1–27, Dec. 2002.
[19] C. L. Borgman, H. Abelson, L. Dirks, R. Johnson, K. R. Koedinger, M. C. Linn,
C. A. Lynch, D. G. Oblinger, R. D. Pea, K. Salen, M. S. Smith, and A. Szalay.
Fostering Learning in the Networked World: The Cyberlearning Opportunity
and Challenge. Report of the NSF Task Force on Cyberlearning, page 64, Sept.
2008.
[20] M. Burnett. Visual programming. Wiley Encyclopedia of Electrical and Electronics
Engineering, 1999.
[21] M. Burnett and M. Baker. A Classification System For Visual Programming Languages. Technical Report 93-60-14, 1994.
[22] M. Burnett, C. Cook, and G. Rothermel. End-user software engineering. Communications of the ACM, 47(9):53–58, Sept. 2004.
139
BIBLIOGRAPHY
[23] S.-K. Chang. Visual Languages: A Tutorial and Survey. Software, IEEE, 4(1):29–
39, Jan. 1987.
[24] A. Cheyer and J. Levy. A Collaborative Programming Environment for Web
Interoperability. 1st Workshop on Semantic Wikis (SemWiki ’06), pages 1–15,
May 2006.
[25] K. Claypool and M. Claypool. Teaching software engineering through game
design. Proceedings of the 10th annual SIGCSE conference on Innovation and
technology in computer science education (ITiCSE ’05), pages 123–127, 2005.
[26] A. Clement. Cooperative support for computer work: a social perspective on the
empowering of end users. Proceedings of the 1990 ACM conference on Computersupported cooperative work, pages 223–236, 1990.
[27] S. Cooper, W. Dann, and R. Pausch. Alice: a 3-D tool for introductory programming concepts. Journal of Computing Sciences in Colleges, 15(5):107–116,
2000.
[28] M. Csikszentmihalyi. Finding Flow: The Psychology Of Engagement With Everyday
Life. Basic Books, Apr. 1998.
[29] A. Cypher. Watch What I Do. Programming by Demonstration. The MIT Press,
May 1993.
[30] A. Cypher and D. C. Smith. KidSim: end user programming of simulations. In
CHI ’95: Proceedings of the SIGCHI conference on Human factors in computing
systems. ACM Press/Addison-Wesley Publishing Co., May 1995.
[31] T. DeMarco and T. R. Lister. Peopleware: productive projects and teams. Dorset
House Pub., page 245, Jan. 1999.
[32] S. Deterding, M. Sicart, L. Nacke, K. O’Hara, and D. Dixon. Gamification: Using
Game Design Elements in Non-Gaming Contexts. CHI ’11 Extended Abstracts on
Human Factors in Computing Systems, pages 2425–2428, 2011.
[33] S. Dossick and G. Kaiser. CHIME: a metadata-based distributed software development environment. ESEC/FSE-7: Proceedings of the 7th European software
engineering conference held jointly with the 7th ACM SIGSOFT international symposium on Foundations of software engineering, Nov. 1999.
[34] S. Downes. E-learning 2.0. eLearn, Oct. 2005.
[35] R. T. Fielding. Architectural styles and the design of network-based software architectures. PhD thesis, University of California, Irvine, 2000.
140
BIBLIOGRAPHY
[36] G. Fischer. Domain-oriented design environments: supporting individual and
social creativity. Computational Models of Creative Design IV, pages 83–111,
1999.
[37] G. Fischer. End-user development and meta-design: Foundations for cultures of
participation. End User Development, pages 3–14, 2009.
[38] G. Fischer and E. Giaccardi. Meta-Design: A Framework for the Future of End
User Development. End User Development. Springer, Dec. 2005.
[39] G. Fischer, E. Giaccardi, Y. Ye, and A. Sutcliffe. Meta-design: a manifesto for
end-user development. Communications of the ACM, 47(9):33–37, Sept. 2004.
[40] G. Fischer, S. Lindstaedt, J. Ostwald, M. Stolze, T. Sumner, and B. Zimmermann.
From domain modeling to collaborative domain construction. DIS ’95: Proceedings of the 1st conference on Designing interactive systems: processes, practices,
methods, & techniques, Aug. 1995.
[41] G. Fischer, K. Nakakoji, and Y. Ye. Metadesign: Guidelines for Supporting Domain Experts in Software Development. Software, IEEE, 26(5):37–44, Sept.
2009.
[42] R. Fournier. Teamwork is the key to remote development. Infoworld, 2001.
[43] B. Freudenberg, Y. Ohshima, and S. Wallace. Etoys for One Laptop Per Child.
In Creating, Connecting and Collaborating through Computing, 2009. C5 ’09.
Seventh International Conference on, pages 57–64, 2009.
[44] A. Gal, B. Eich, M. Shaver, D. Anderson, D. Mandelin, M. R. Haghighat, B. Kaplan, G. Hoare, B. Zbarsky, J. Orendorff, J. Ruderman, E. W. Smith, R. Reitmaier,
M. Bebenita, M. Chang, and M. Franz. Trace-based just-in-time type specialization for dynamic languages. In Proceedings of the 2009 ACM SIGPLAN conference on Programming language design and implementation, pages 465–478, New
York, NY, USA, 2009. ACM.
[45] E. Gamma, R. Helm, R. Johnson, and J. Vlissides. Design Patterns: Elements of
Reusable Object-Oriented Software. Addison-Wesley, 1 edition, 1995.
[46] M. Gantt and B. Nardi. Gardeners and gurus: patterns of cooperation among
CAD users. CHI ’92: Proceedings of the SIGCHI conference on Human factors in
computing systems, June 1992.
[47] J. P. Gee. What Video Games Have to Teach Us About Learning and Literacy.
Palgrave Macmillan, 2003.
[48] J. P. Gee. Good video games + good learning. collected essays on video games,
learning, and literacy. Peter Lang Pub Inc, 2007.
141
BIBLIOGRAPHY
[49] C. Ghezzi, M. Jazayeri, and D. Mandrioli. Fundamentals of Software Engineering.
Prentice Hall, 2 edition, Sept. 2002.
[50] A. Goldberg. Smalltalk-80: The Interactive Programming Environment. 1984.
[51] J. Grudin. Computer-Supported Cooperative Work: History and Focus. Computer, 27(5):19–26, 1994.
[52] E. R. Hayes and I. A. Games. Making Computer Games and Design Thinking:
A Review of Current Software and Strategies. Games and Culture, 3(3-4):309–
332, 2008.
[53] P. Henriksen and M. Kölling. greenfoot: combining object visualisation with interaction. In OOPSLA ’04: Companion to the 19th annual ACM SIGPLAN conference on Object-oriented programming systems, languages, and applications, 2004.
[54] M. A. Honey and M. Hilton. Learning Science Through Computer Games and
Simulations. The National Academies Press, page 180, Jan. 2011.
[55] P. Hudak. Building domain-specific embedded languages. ACM Computing Surveys, 28(4es), Dec. 1996.
[56] A. Ioannidou, V. Bennett, A. Repenning, K. H. Koh, and A. Basawapatna. Computational Thinking Patterns. Annual Meeting of the American Educational Research
Association, page 15, Apr. 2011.
[57] C. Jones. Programming productivity. page 280, Jan. 1986.
[58] Y. B. Kafai and M. Resnick. Constructionism in practice. designing, thinking, and
learning in a digital world. Lawrence Erlbaum, Apr. 1996.
[59] A. Kay. Squeak Etoys, Children & Learning. VPRI Research Note, 2005.
[60] C. Kazimoglu, M. Kiernan, L. Bacon, and L. MacKinnon. Learning Programming
at the Computational Thinking Level via Digital Game-Play. Procedia Computer
Science, 9:522–531, Jan. 2012.
[61] C. Kelleher and R. Pausch. Lowering the barriers to programming: A taxonomy
of programming environments and languages for novice programmers. ACM
Computing Surveys, 37(2):83–137, June 2005.
[62] C. Kelleher and R. Pausch. Using storytelling to motivate programming. Communications of the ACM, 50(7), July 2007.
[63] A. J. KO, R. Abraham, L. Beckwith, A. Blackwell, M. Burnett, M. Erwig, C. Scaffidi, J. Lawrance, H. Lieberman, B. Myers, M. B. Rosson, G. Rothermel, M. Shaw,
and S. Wiedenbeck. The state of the art in end-user software engineering. ACM
Computing Surveys, 43(3):1–44, Apr. 2011.
142
BIBLIOGRAPHY
[64] K. H. Koh, A. Basawapatna, V. Bennett, and A. Repenning. Towards the Automatic Recognition of Computational Thinking for Adaptive Visual Language
Learning. IEEE Symposium on Visual Languages and Human-Centric Computing(VL/HCC), pages 59–66, 2010.
[65] R. Kop and H. Fournier. New Dimensions to Self-Directed Learning in an Open
Networked Learning Environment. International journal of Self-Directed Learning, 7(2), 2011.
[66] J. Lave and E. Wenger. Situated learning: legitimate peripheral participation.
Cambridge University Press, 1991.
[67] B. Leuf and W. Cunningham. The Wiki way: quick collaboration on the Web.
Addison-Wesley, 2001.
[68] C. Lewis and A. Repenning. Creating Educational Gamelets. Educating Learning
Technology Designers: Guiding and Inspiring Creators of Innovative Educational
Tools, C. DiGiano, S. Goldman, and M. Chorost, Eds. New York: Routledge, pages
203–229, 2008.
[69] H. Lieberman, F. Paternó, M. Klann, and V. Wulf. End-User Development: An
Emerging Paradigm. In H. Lieberman, F. Paterno, and V. Wulf, editors, End User
Development. Springer, 2006.
[70] W. Mackay. Patterns of sharing customizable software. CSCW ’90: Proceedings of
the 1990 ACM conference on Computer-supported cooperative work, Sept. 1990.
[71] J. Mackness, S. F. J. Mak, and R. Williams. The ideals and reality of participating in a MOOC . Proceedings of the 7th International Conference on Networked
Learning 2010, pages 266–274, 2010.
[72] M. MacLaurin. Kodu: end-user programming and design for games. In Proceedings of the 4th International Conference on Foundations of Digital Games, page
xviii, New York, New York, USA, 2009. ACM Press.
[73] M. MacLaurin. The Design of Kodu: a Tiny Visual Programming Language for
Children on the Xbox 360. POPL’11, Jan. 2011.
[74] A. MacLean, K. Carter, L. Lövstrand, and T. Moran. User-tailorable systems:
pressing the issues with buttons. Proceedings of the SIGCHI conference on Human
factors in computing systems: Empowering people, Dec. 1989.
[75] T. Malone, K. Grant, K. Lai, R. Rao, and D. Rosenblitt. Semistructured messages
are surprisingly useful for computer-supported coordination. ACM Transactions
on Office Information Systems, Dec. 1986.
143
BIBLIOGRAPHY
[76] Marina Papastergiou. Digital Game-Based Learning in high school Computer
Science education: Impact on educational effectiveness and student motivation.
Computers & Education, 52(1):1–12, Jan. 2009.
[77] J. McGonigal. Reality Is Broken. Why Games Make Us Better and How They Can
Change the World. The Penguin Press, 2011.
[78] C. McLoughlin and M. J. Lee. Social software and participatory learning: Pedagogical choices with technology affordances in the Web 2.0 era. In Proceedings
ASCILITE Singapore 2007, pages 664–675, 2007.
[79] M. Mernik, J. Heering, and A. Sloane. When and how to develop domainspecific languages. ACM Computing Surveys, 37(4):316–344, 2005.
[80] N. Meyrowitz. The missing link: why we’re all doing hypertext wrong. MIT Press,
Nov. 1989.
[81] D. Michael and S. Chen. Serious Games: Games That Educate, Train, and Inform.
Course Technology PTR, 1 edition, Oct. 2005.
[82] J. Michael. In Pursuit of Meaningful Learning. Advances in Physiology Education,
25(3):145–158, 2001.
[83] A. Mørch. Three levels of end-user tailoring: customization, integration, and
extension. In Computers and design in context, pages 51–76. The MIT Press, Oct.
1997.
[84] A. Mørch and N. Mehandjiev. Tailoring as Collaboration: The Mediating Role of
Multiple Representations and Application Units. Computer Supported Cooperative Work (CSCW), Dec. 1999.
[85] P. Moreno-Ger, D. Burgos, I. Martínez-Ortiz, J. L. Sierra, and B. FernándezManjón. Educational game design for online education. Computers in Human
Behavior, 24(6):2530–2540, Sept. 2008.
[86] B. Myers, S. Hudson, and R. Pausch. Past, present, and future of user interface software tools. ACM Transactions on Computer-Human Interaction (TOCHI),
7(1):3–28, 2000.
[87] B. Nardi. A Small Matter of Programming: Perspectives on End User Computing.
MIT Press, Dec. 1992.
[88] B. Nardi and J. Miller. Twinkling lights and nested loops: distributed problem
solving and spreadsheet development. International Journal of Man-Machine
Studies, Dec. 1990.
144
BIBLIOGRAPHY
[89] J. Nielsen. Response Times: the Three Important Limits. In Usability Engineering. Morgan Kaufmann, 1993.
[90] H. L. Oppenheimer. Project Management Issues in Globally Distributed Development. Proceedings of the Global Software Development Workshop, 2002.
[91] T. O’Reilly. What is Web 2.0 - Design patterns and business models for the next
generation of software. 2005.
[92] P. Oreizy and G. Kaiser. The Web as enabling technology for software development and distribution. Internet Computing, IEEE, 1(6):84–87, 1997.
[93] M. Overmars. Teaching computer science through game design. Computer,
37(4):81–83, 2004.
[94] S. Papert. Mindstorms: children, computers, and powerful ideas. Mindstorms:
children, computers, and powerful ideas, Jan. 1980.
[95] V. Pipek and H. Kahler. Supporting Collaborative Tailoring. End User Development. Springer, Dec. 2004.
[96] M. Prensky. Digital game-based learning. Computers in Entertainment (CIE),
1(1):21, 2003.
[97] M. Prensky. Don’t Bother Me Mom–I’m Learning! Paragon House Publishers,
Mar. 2006.
[98] A. Ravenscroft. Social software, Web 2.0 and learning: status and implications
of an evolving paradigm. Journal of Computer Assisted Learning, 25(1):1–5, Jan.
2009.
[99] E. S. Raymond. The Cathedral and the Bazaar-Musings on Linux and Open
Source by an Accidental Revolutionary. O’Reilly, 1999.
[100] M. Reed, A. C. Evely, G. Cundill, I. R. A. Fazey, J. Glass, A. Laing, J. Newig,
B. Parrish, C. Prell, C. Raymond, and L. C. Stringer. What is social learning?
Ecology and Society, 2010.
[101] A. Repenning. Agentsheets: a Tool for Building Domain-Oriented Dynamic, Visual
Environments. PhD thesis, University of Colorado, 1993.
[102] A. Repenning. Making programming more conversational. Visual Languages and
Human-Centric Computing (VL/HCC), 2011 IEEE Symposium on, pages 191–194,
2011.
[103] A. Repenning. Programming goes back to school. Commun. ACM, 55(5):38–40,
2012.
145
BIBLIOGRAPHY
[104] A. Repenning and J. Ambach. Tactile programming: a unified manipulation
paradigm supporting program comprehension, composition and sharing. Proceedings of the 1996 IEEE Symposium on Visual Languages, pages 102–109, Aug.
1996.
[105] A. Repenning and J. Ambach. Tactile programming: A unified manipulation
paradigm supporting program comprehension, composition and sharing. Proceedings of the 1996 IEEE Symposium on Visual Languages, page 102, 1996.
[106] A. Repenning and J. Ambach. Visual Agentalk: Anatomy of a Low Threshold,
High Ceiling End User Programming Environment. Technical Report, page 12,
1996.
[107] A. Repenning and J. Ambach. The agentsheets behavior exchange: supporting social behavior processing. CHI’97 extended abstracts on Human factors in
computing systems: looking to the future, pages 26–27, 1997.
[108] A. Repenning and A. Ioannidou. AgentCubes: Raising the Ceiling of EndUser Development in Education through Incremental 3D. Visual Languages and
Human-Centric Computing (VL/HCC), IEEE Symposium on, pages 27–34, 2006.
[109] A. Repenning and A. Ioannidou. Broadening Participation through Scalable
Game Design. SIGCSE ’08: Proceedings of the 39th SIGCSE technical symposium
on Computer science education, Feb. 2008.
[110] A. Repenning and T. Sumner. Agentsheets: A medium for creating domainoriented visual languages. Computer, 28(3):17–25, 1995.
[111] A. Repenning, D. Webb, and A. Ioannidou. Scalable game design and the development of a checklist for getting computational thinking into public schools.
In SIGCSE ’10: Proceedings of the 41st ACM technical symposium on Computer
science education, 2010.
[112] M. Resnick, B. Silverman, Y. Kafai, J. Maloney, A. Monroy-Hernández, N. Rusk,
E. Eastmond, K. Brennan, A. Millner, E. Rosenbaum, and J. Silver. Scratch:
Programming for All. Communications of the ACM, 52(11):60–67, Nov. 2009.
[113] L. Rich, H. Perry, and M. Guzdial. A CS1 course designed to address interests of
women. In SIGCSE ’04: Proceedings of the 35th SIGCSE technical symposium on
Computer science education, 2004.
[114] V. Rideout, D. Roberts, and U. Foehr. Generation M: Media in the Lives of 8-18
year-olds, Executive Summary. A Kaiser Family Foundation Study, pages 1–41,
Jan. 2005.
146
BIBLIOGRAPHY
[115] D. E. Rowley and D. G. Rhoades. The cognitive jogthrough: a fast-paced user
interface evaluation procedure. In CHI ’92: Proceedings of the SIGCHI conference
on Human factors in computing systems, 1992.
[116] A. Sarma. A Survey of Collaborative Tools in Software Development . University
of California, Irvine, Institute for Software Research, Technical Report, page 68,
Mar. 2005.
[117] S. Sawyer. Software development teams. Communications of the ACM, 47(12),
Dec. 2004.
[118] C. Scaffidi and C. Chambers. Skill progression demonstrated by users in the
Scratch animation environment. International Journal of Human-Computer Interaction, 8(6):383–398, Apr. 2012.
[119] C. Scaffidi, M. Shaw, and B. Myers. Estimating the numbers of end users and end
user programmers. IEEE Symposium on Visual Languages and Human-Centric
Computing, pages 207–214, 2005.
[120] A. Settle. Computational thinking in a game design course. Proceedings of the
2011 conference on Information technology education, pages 61–66, 2011.
[121] M. Shaw. Software engineering education: a roadmap. Proceedings of the conference on The future of Software Engineering (ICSE ’00), pages 371–380, 2000.
[122] B. Shneiderman. Direct Manipulation: A Step Beyond Programming Languages.
Computer, 16(8):57–69, Aug. 1983.
[123] B. Shneiderman. Response time and display rate in human performance with
computers. ACM Computing Surveys, 16(3):265–285, Sept. 1984.
[124] N. C. Shu. Visual programming. Van Nostrand Reinhold, 1988.
[125] D. C. Smith, A. Cypher, and J. Spohrer. KidSim: programming agents without a
programming language. Communications of the ACM, 37(7), July 1994.
[126] R. Stallman. EMACS the extensible, customizable self-documenting display editor. ACM SigPlan Notices, Dec. 1980.
[127] J. Strubing. Designing the Working Process: What Programmers Do Besides Programming. User-Centered Requirements for Software Engineering Environments,
Springer, Berlin, Germany, 1994.
[128] K. Sung. Computer Games and Traditional CS Courses. Communications of the
ACM, 52(12), Dec. 2009.
147
BIBLIOGRAPHY
[129] D. Tapscott. Grown Up Digital: How the Net Generation is Changing Your World.
McGraw-Hill, 1 edition, Oct. 2008.
[130] D. Teague and P. Roe. Collaborative learning: towards a solution for novice
programmers. Proceedings of the tenth conference on Australasian computing
education-Volume 78, pages 147–153, 2008.
[131] G. Toffetti, S. Comai, J. Preciado, and M. Linaje. State-of-the Art and Trends
in the Systematic Development of Rich Internet Applications. Journal of Web
Engineering, 10(1):70–86, 2011.
[132] J.-P. Tolvanen, J. Gray, and M. Rossi. 2 nd workshop on domain-specific visual
languages. ACM, New York, New York, USA, Nov. 2002.
[133] M. Turner, D. Budgen, and P. Brereton. Turning software into a service. Computer, 36(10):38–44, Oct. 2003.
[134] A. van Deursen, P. Klint, and J. Visser. Domain-specific languages: An annotated
bibliography. ACM SigPlan Notices, 35(6), June 2000.
[135] J. Vassileva. Toward Social Learning Environments. IEEE Transactions on Learning Technologies, 1(4):199–214, 2008.
[136] G. Vickery, S. Wunsch-Vincent, and OECD. Participative Web and User-Created
Content: Web 2.0, Wikis and Social Networking. 2007.
[137] A. Voutilainen. Part-of-Speech Tagging. In R. Mitkov, editor, The Oxford Handbook of Computational Linguistics, pages 219–232. Oxford University Press, Jan.
2003.
[138] G. Wang, S. Yang, and Y. Han. Mashroom: End-User Mashup Programming
Using Nested Tables. Proceedings of the 18th International Conference on World
Wide Web, pages 861–870, Dec. 2008.
[139] E. Wenger. Communities of Practice: Learning, Meaning, and Identity. Cambridge
Univ Press, Sept. 1999.
[140] K. Werbach and D. Hunter. For the Win: How Game Thinking Can Revolutionize
Your Business. Wharton Digital Press, Oct. 2012.
[141] L. Werner, J. Denner, S. Campe, and D. C. Kawamoto. The fairy performance
assessment: measuring computational thinking in middle school. In SIGCSE ’12:
Proceedings of the 43rd ACM technical symposium on Computer Science Education,
2012.
[142] E. J. J. Whitehead and M. Wiggins. WebDAV: IEFT standard for collaborative
authoring on the Web. Internet Computing, IEEE, 2(5):34–40, 1998.
148
BIBLIOGRAPHY
[143] J. Whitehead. Collaboration in Software Engineering: A Roadmap. Future of
Software Engineering (FOSE ’07), Dec. 2006.
[144] L. Williams, E. Wiebe, K. Yang, M. Ferzli, and C. Miller. In Support of Pair
Programming in the Introductory Computer Science Course. Computer Science
Education, 12(3):197–212, Jan. 2002.
[145] C. Wills, D. Finkel, M. Gennert, and M. Ward. Peer learning in an introductory
computer science course. ACM SIGCSE Bulletin, 26(1):309–313, Mar. 1994.
[146] C. Wilson, L. A. Sudol, C. Stephenson, and M. Stehlik. Running on empty:
The failure to teach K-12 computer science in the digital age. Association for
Computing Machinery/Computer Science Teachers Association, 2010.
[147] J. M. Wing. Computational thinking. Communications of the ACM, 49(3), Mar.
2006.
[148] M. Wu and K. Richards. Facilitating computational thinking through game
design. Edutainment Technologies. Educational Games and Virtual Reality/Augmented Reality Applications, pages 220–227, 2011.
[149] W. Xiao, C. Chi, and M. Yang. On-line collaborative software development via
wiki. WikiSym ’07: Proceedings of the 2007 international symposium on Wikis,
Oct. 2007.