Session 10 July 2013
Towards a framework for games and simulations in STEM subject assessments
Gary Wills (University of Southampton, UK).
Session spoken by Ashok Ranchhod (University of Southampton, UK).
Please test your audio set up using the wizard:
Tools ->Audio -> Audio setup wizard.
Your Hosts
Professor Geoff Crisp,
Dean Learning and Teaching, RMIT University geoffrey.crisp@rmit.edu.au
Dr Mathew Hillier,
Teaching and Educational Development Institute, University of Queensland mathew.hillier@uq.edu.au

Talk by Ashok Ranchhod
Acknowledgement:
Gary Wills, James Stallwood, James Baker, Lester Gilbert, Alejandra
Recio Saucedo, Alex Frazer, Teresa Binks

Nobody is really sure…
“A series of meaningful choices” – Sid Meier, designer
“Civilization”
“One or more causally linked series of challenges in a simulated
environment” – Ernest Adams & Andrew Rollings, game design authors
“Conflicts in which players work to foil each other’s goals” –
Chris Crawford, game designer and theorist
“A system in which players engage in an artificial conflict,
defined by rules, that results in a quantifiable outcome” – Katie
Salen & Eric Zimmerman, game design authors

“a rule-based formal system with a variable and
quantifiable outcome, where different outcomes are assigned different values, the player exerts effort in order to influence the outcome, the player feels attached to the outcome, and the consequence of the activity are optional and negotiable”
– Jesper Juul, video game theorist
Choices, simulated environments, conflicts, goals, rules, engagement, rewards

Creating a game concept, then defining how that game works
Defining the underlying rules and behaviours which govern the game world
This condition can be completed competitively or cooperatively
Or both!
Rules set games apart from toys and roleplay sets them aside from puzzles
Weaving an engaging narrative into the game
Deciding how the player will interact with the game world

• Initially, games were used for educational purposes for purely motivational reasons
– Games are fun, if we make a learning game, it’ll make the learning fun
– Not necessarily true…

• Knowledge is arguably useless in isolation
– We need to be able to apply it to a given context
• Games will typically introduce new skills and ideas in isolation
– In tutorials at the beginning of the game, or at various points throughout
• Once these skills are taught, the player will be called upon to use them in progressively more complex scenarios
– As more skills are learnt, the more complex the scenarios can become…

Maintaining the “Flow” zone
Make game challenges difficult enough to stop the player becoming bored, without being so difficult as to be impossible
Difficulty curve
The player is likely to become better at a game as they progress
Challenges will become relatively easier
If they become too easy, the player will grow bored
Increase the difficulty accordingly, to maintain a state of
Flow
The rate of change of this curve can be difficult to engineer!

Feedback loops
The rewards a player receives for completing a task could make it easier to complete the same task in future…
…earning even more rewards…
…making the task even easier still!
This can make the game boring (or frustratingly difficult for a multiplayer opponent), disrupting the Flow state
We can overcome this in a number of ways
Make the reward less powerful
Gradually make the reward less powerful as the player progresses
Offer rewards for one task which help another

Understanding and the construction of knowledge comes through learners undertaking
learning activities and receiving feedback from taking the activity.
In some subjects this will include serious games, simulations, and the building and running of models.

Learning activities inherently engage and motivate the learner through their characteristic combination of learning actions and consequential feedback.
These are part of the wider formative learning activates used to motivate, and provide a contextual understanding of the subject domain.

Providing games and simulations to address specific educational objectives in different subject areas may require custom-built applications
That are difficult-to-share difficult-to-reuse solutions whose effectiveness is highly dependent upon the personal skills of specialist developers

During the past decades, researchers have explored the effectiveness and impact of games that can be used as educational tools and in supporting learning of STEM at any educational level: facilitate students’ inquiry, encourage them to engage in “What if ” explorations, allow multiple and dynamic external visualisations through multimedia support, optimally communicate STEM abstract, complex and nonperceptible concepts, promote metacognitive skills, and support social interaction and collaborative learning.

A pedagogically informed learning activity is based upon the existing competence of the learner, is coupled with prerequisite competences, and seeks to develop one or more articulated desired competences.

A pedagogically informed (top level) competence is conceived as an acyclic directed graph of enabling competences
Each competence (top level and enabling) comprising a contextualised intended learning outcome (ILO).
An ILO in turn comprises a learned capability with respect to a specified topic or item of subject matter.
A number of PhDs have been and are being conducted in the area of computer based competence structures in the group

Pedagogically informed teaching activities associated with the specified learning activities include: the provision of appropriate materials, the assessment of learner performance on the desired competences, the provision of feedback which is well-timed, contingent, and specific.

Courtesy of Lester Gilbert

Subject
Content
Fact
Concept
Procedure
Principle
E-learning Objective
Remember
Instance
Remember
Generally
Use Find
Example
Name
Rehearse
Explanation
Explain
State definition
Explain steps
State cause effect relationship
Classify
Demonstrates
Demonstratio ns
Predict
Explore categories
Invent
Definition
Explore procedure
Devises procedure
Explore problem
Discover principle

ILOs are a an educational way of setting an achievable and measurable goal.
Goals are useful for learner motivation
Game designers consider them equally important
“The 400 List” puts “provide clear short-term goals” and “provide an enticing long-term goal” at positions 6 and 25 respectively
Long term goals can be used to imply a context for more immediate, short-term goals

The question we asked was how can we construct and then take a machine-processable expression of an educational intention and (either automatically or semi-automatically) turn it into a computerdeliverable serious game, simulation, or adaptive formative assessment with (some elements of) pedagogical validity?
As a response, we propose a framework for
Pedagogically Effective Games & Simulations
(PEGS).

In such a framework, learners will be able to assess their current learning needs and navigate a machinereadable ILO structure to meet their learning goals.
Each ILO will be associated with one or more learning activities.
Where a learning activity is designed to feed the curiosity and reasoning processes of the learner, engagement and motivation is likely, allowing the learner to gain understanding and construct knowledge through their learning activity and consequential feedback.

The pedagogical strategy is to invoke active learner participation supported by multiple resources made available in the learning environment.
The learner participation involves searching for, evaluating, and using authentic information.
This learning experience mimics real life in targeting the learner as the routine information hunter and interpreter who constructs knowledge by problem solving with information tools.
The advantages to this strategy are the adoption of a studentcentred approach to learning and the promotion of thinking skills (problem solving, reasoning, and critical evaluation).

A framework should allow for the 'routine' construction of cost-effective, shareable and reusable games and simulations as pedagogically-informed learning activities by teachers and learners who do not necessarily have gaming experience or skills.
In the same way that that learning materials are routinely constructed in support of one or more ILOs and assessments are routinely constructed to test the achievement of one or more ILOs.
The framework should use open standards in the creation of games and simulations, thus ensuring continuous development and uptake by interested communities

The ‘PEGS-I’ tool is an application which supports a teacher in identifying and structuring intended student competences from syllabus and curriculum descriptions, from teacher expectations and desires, and possibly from national and international prescriptions.
‘PEGS-D’ is a standards-defined interoperable competences database which represents the topics of interest to a particular community.
Visualising the PEGS-D competences uses the ‘PEGS-V’ visualisation tool, which also supports exchanging PEGS-D competences with others, tagging key competences with supporting learning and teaching materials, etc.

One or more game and simulation templates are populated from PEGS-D to generate the desired learning activities using
‘PEGS-E’, the Editor.
Templates may be pre-defined or developed from a ‘blank sheet’ or by adapting existing templates. The templates are standards-defined and interoperable
(IMS-LD, IMS LTI, with extensions), yielding games and simulations which can be played using a compatible software application, the PEGS-P player. The games and simulations are held in the PEGS-GS repository.
The simpler games and simulations are single-user, the more advanced multi-player.

Assessments are generated using the ‘PEGS-A’ (assessment) tool and are held in a compatible repository, PEGS-X, as a standards-defined and interoperable (IMS-QTI, with possible extensions, and/or IMS LTI) file.
Finally, we envision a future ‘PEGS-ID’ tool to provide suggestions for teaching strategies and instructional designs which are appropriate to constituent competence sub-graphs in the topic, and an intelligent tutor, ‘PEGS-ITS’, to adaptively deliver materials and assessments.

The framework will be delivered through a series of easily adoptable tool-kits.
Toolkits “provide a pragmatically-based approach to applying theory to practice” and can be used to support decisionmaking. (Conole et al., 2005)
Enable the 'routine' construction of cost-effective, shareable and reusable games and simulations as pedagogicallyinformed learning activities by teachers and learners,
Similar to the way that learning materials are routinely constructed in support of one or more ILOs and assessments are routinely constructed to test the achievement of one or more ILOs.

Maintaining the flow in an Educational Game
How can we use the underlying competence structure to automatically provide the balance of a game.
Visualizing the graphs structure
Mapping the competence structures in such a way that learners can map out their own route for learning.

Connecting Games to competences
What affordances do games offer and how can we relate these to the competences we require from learners.
In a collaborative world what affordances can
MMROPGs offer education.
How can the learning know (trust) if an activity will meet the goals.
Automatic adaption based on the experience of the user.

Navigating the competence structure
Obtain appropriate learning activities for each of the competences and path to take to meet their goals.
Work by Onjira Sitthisak and Lester Gilbert suggested that we can automatic generate assessment questions from the competency tree
Can we build a list of automatic activates to meet a given competence
Feedback: can we use the structure to give automatic feedback

EUFLORIA-Strategy Game on colonising the earth

PROTEUS-A Sandbox game-explore enjoy learn
Can these be formalised and competencies worked out?

CONTENT
ACTING
Win
Build
Design Create
Purchase
Decorate EXPRESS
Customise
Choose
Challenge
COMPETE
Showoff
Compare Taunt
PLAYERS
View Collect Rate
EXPLORE
Vote
Curate Review
Comment
Like
COLLABORATE Greet
Help Share
Give
INTERACTING

Games for learning have the potential to engage and motivate learning
They are fun
They can provide (instance) feedback
They require clear learning goals
They can maintain the flow
They can provoke curiosity (what if scenarios )
They can be played individually or within a group and you can collaborate or compete.

We have presented a framework to support the
'routine' development of effective games and simulations from pedagogically-informed statements of intended learning outcomes.
That would be machine-processable turn it into a computer-deliverable serious game, simulation, or adaptive formative assessment with pedagogical validity.

Webinar Feedback
Please kindly fill in the feedback survey by clicking this link:
With Thanks from your hosts
Professor Geoff Crisp,
Dean Learning and Teaching, RMIT University geoffrey.crisp@rmit.edu.au
Dr Mathew Hillier,
Teaching and Educational Development Institute, University of Queensland mathew.hillier@uq.edu.au