Generalizability of Goal Recognition
Models in
Narrative-Centered Learning
Environments
Alok Baikadi
Bradford Mott
Jonathan Rowe,
James Lester
North Carolina State University
1
Goal Recognition in
Narrative-Centered Learning
Environments
 Task: Identify the specific
objective that the player
is attempting to achieve
 Goal recognition models
enable the following:
• Preemptively augmenting
narrative experiences
• Assessing problem solving
in narrative-centered
learning environments
• Iteratively refining learning
environment
2
Generalization of Goal
Recognition
 Goal Recognition is typically very domain
dependent
• Plan libraries
• Many domain-independent techniques are only
evaluated on one domain
 Research Question: Can a domain-specific goal
recognition model be applied in a principled way
to a new domain and achieve similar results?
3
Related Work

Goal recognition is a restricted form of plan recognition (Carberry 2001; Kautz & Allen, 1986;
Singla & Mooney, 2011)

Investigated widely in numerous domains (Blaylock & Allen, 2003; Charniak & Goldman, 1993; Lesh, Rich
& Sidner, 1999)

IO-HMM approach for recognizing high-level goals in simple action-adventure
game (Gold, 2010)

PHATT-based approach for behavior recognition in real-time strategy game (Kabanza,
Bellefeuille & Bisson, 2010)

N-gram and Bayesian network approaches for goal recognition to support dynamic
narrative planning (Mott, Lee & Lester, 2006)

MLN-based approaches (Singla and Mooney, 2011 ; Ha et al., 2011 ; Sadilek and Kautz, 2012)
4
Outline
 Goal Recognition Approach
 Goal Recognition Corpora
 Evaluation & Discussion
 Conclusions and Future Work
5
Markov Logic Networks (MLNs)
 Statistical relational learning
• Combines first-order relational reasoning with statistical
learning
• Input: A set of first-order predicate calculus formulae, along
with weights
• Formulae can be expanded into a Markov Random Field for
learning and inference
 The joint probability distribution is defined as:
1
P(X = x) = Pk (F(xk ))
Z
 Toolkit: Markov TheBeast (Riedel, 2008)
6
Representation
Predicate
Interpretation
action(t, a)
Action a happens at time t
argument(t, a)
Argument a observed at time t
location(t, l)
Player is at location l at time t
state(t, s)
The narrative is in state s at time t
goal(t, g)
Player is pursuing goal g at time t
7
Context in Goal Recognition
 Actions are not always independent
 Traditional goal recognition formulation allows
for all previous observations
• Can lead to sparsity issues
 Solution: Look for key events in the history
that provide insight to the player’s context
 Use the structure of the narrative to provide
the context
8
Discovery Events
 Task progress is represented by a sequence of
discovery events
 Partial Answers to Central Questions are clues
towards the solution
 Provides a context for goal recognition: What
has the user discovered?
9
Discovery Event Formulae
 Milestone formulae recognize which discovery events
have already occurred
 Uses a cardinality constraint to capture existence
10
Outline
 Goal Recognition Approach
 Goal Recognition Corpora
 Evaluation & Discussion
 Conclusions and Future Work
11
CRYSTAL ISLAND: OUTBREAK
 8th grade
microbiology
 Valve
Software’s
Source engine
 Science
mystery
 Goal: Identify
source and
treatment of
outbreak
12
CRYSTAL ISLAND:
Introduction
1.
Student plays the role of a new
visitor to the island.
2.
Student discovers that several
team members have fallen sick.
13
CRYSTAL ISLAND:
Gathering Information
3.
Student gathers clues from sick
team members.
4.
Student asks the camp’s
pathogen experts about
microbiology concepts.
14
CRYSTAL ISLAND:
Gathering Information
5.
Student views microbiologythemed posters.
6.
Student reads books about
different types of pathogens.
15
CRYSTAL ISLAND:
Hypothesis Testing
7.
Student conducts tests using
laboratory equipment.
8.
Student interacts with the lab
technician to view microscopic
images of pathogens.
16
CRYSTAL ISLAND:
Reporting Findings
9.
Student presents findings and recommended
treatment to camp nurse.
17
Corpus Collection
 Eighth grade class from
public middle school
 153 participants
 No prior experience with
CRYSTAL ISLAND
 Played game for 1 hour, or
until they were finished
 7 goals available to students
(Ha et al., 2011)
18
CRYSTAL ISLAND: UNCHARTED DISCOVERY
Upper Elementary Science
Subject
 5th grade science
 Standards aligned
Content
 Landforms
 Maps, models &
navigation
Story
 Adventurous
adolescent
 Shipwrecked crew
 Complete quests to
explore island
19
CRYSTAL ISLAND: UNCHARTED DISCOVERY
Video
Video
20
Corpus Collection
 Onsite at 8 schools
 831 fifth grade students
 62% Caucasian, 14%
African American, 8%
Asian, 16% Other
 Teacher-driven
implementation in
classrooms
 6 one hour sessions over
4 weeks
 12 goals available during
the first 2 weeks
21
Goal Extraction Procedure
 Goal-achieving actions were
identified
 Actions between previous goal and
current goal were labeled with
current goal
 Goal-achieving actions were
removed
22
Outline
 Goal Recognition Approach
 Goal Recognition Corpora
 Evaluation & Discussion
 Conclusions and Future Work
23
Empirical Evaluation
 State of the Art Baseline:
• Factored model (Ha et al., 2011)
• Uses MLNs to relate the current time step to the previous time step
 Each model was evaluated using 10-fold student-level
cross-validation
 Each model was evaluated according to three metrics:
• Accuracy: Measured as F1 score
• Convergence rate: Percent of sequences which eventually
predicted the correct goal
• Convergence point: In sequences that converged, the percent of
actions that had to be observed before a consistent prediction
was made
24
Baseline Model
25
CRYSTAL ISLAND: OUTBREAK
Discovery Events Model
26
CRYSTAL ISLAND: UNCHARTED DISCOVERY
Discovery Events
27
EXPERIMENTAL RESULTS
Crystal Island: Outbreak
Model
F1
Convergence
Rate
Convergence
Point
Baseline
0.488
30.906
50.865
Discovery Events
0.546
50.056
35.862
Crystal Island: Uncharted Discovery
Model
F1
Convergence
Rate
Convergence
Point
Baseline
0.226
11.915
87.786
Discovery Events
0.244
29.973
79.350
28
Outline
 Goal Recognition Approach
 Goal Recognition Corpora
 Evaluation & Discussion
 Conclusions and Future Work
29
Conclusions
 Goal recognition models show considerable promise
for enhancing the effectiveness of narrative-centered
learning environments
 Encoding narrative discovery events in Markov Logic is
a natural approach for representing context for student
actions in goal recognition
 Experimental findings from two narrative-centered
learning environments suggest that narrative discovery
events enhance the accuracy and convergence of stateof-the-art MLN-based goal recognition models.
30
Future Work
 Investigate combinations of discovery events
• Some of the milestones may have provided more
information than others
• Use automated feature selection
 Integrate goal recognition into a runtime environment
• Can establish intuition for how accurate a model is
necessary
 Elicit feedback from player
• Assumes goals achieved are intended
• May cause some bias
31
Collaborators
Research Staff
Eleni Lobene
Rob Taylor
Affiliated Faculty
Carol Brown (East Carolina University)
Roger Conner (East Carolina University)
Patrick FitzGerald (Art + Design)
Elizabeth Hodge (East Carolina University)
James Minogue (Elementary Education)
John Nietfeld (Educational Psychology)
Marc Russo (Art + Design)
Hiller Spires (Curriculum & Instruction)
Eric Wiebe (STEM Education)
Postdoc
Eunyoung Ha
Digital Art Staff
Kirby Culbertson
Sarah Hegler
Karoon McDowell
Graduate Students
Julius Goth
Joe Grafsgaard
Eunyoung Ha
Seung Lee
Sam Leeman-Munk
Wookhee Min
Chris Mitchell
Jennifer Sabourin
Andy Smith
Undergraduate Student
Stephen Cossa
Affiliated Post-Docs and Graduate Students
(Art, Education, Psychology)
Megan Hardy (Human Factors)
Kristin Hoffman (Educational Psychology)
Angela Meluso (Curriculum & Instruction)
Lucy Shores (Educational Psychology)
Sinky Zheng (Curriculum & Instruction)
32
Acknowledgments
Support provided by the National Science Foundation under grant DRL-0822200.
Additional support was provided by the Bill and Melinda Gates Foundation, the William
and Flora Hewlett Foundation, EDUCAUSE, and the Social Sciences and Humanities
Research Council of Canada.
33
34
Goal Recognition
Example
35
Goal Recognition Example
Crystal Island Virtual Environment Map
Living
Quarters
Infirmary
Waterfall
Camp
Entrance
Bryce’s
Quarters
Dining
Hall
Waterfall:
A nice place to cool off.
Bryce’s Quarters: Living quarters for Bryce, lead scientist.
Laboratory
Laboratory:
Research lab facilities
Living Quarters: Living quarters for team members
36
Goal Recognition Example
Crystal Island Virtual Environment Map
Living
Quarters
Infirmary
Waterfall
Camp
Entrance
Bryce’s
Quarters
Dining
Hall
Waterfall:
A nice place to cool off.
Bryce’s Quarters: Living quarters for Bryce, lead scientist.
Laboratory
Laboratory:
Research lab facilities
Living Quarters: Living quarters for team members
37
Goal Recognition Example
Crystal Island Virtual Environment Map
Living
Quarters
Infirmary
Waterfall
Camp
Entrance
Bryce’s
Quarters
Dining
Hall
Waterfall:
A nice place to cool off.
Bryce’s Quarters: Living quarters for Bryce, lead scientist.
Laboratory
Laboratory:
Research lab facilities
Living Quarters: Living quarters for team members
38
Goal Recognition Example
Crystal Island Virtual Environment Map
Living
Quarters
Infirmary
Waterfall
Camp
Entrance
Bryce’s
Quarters
Dining
Hall
Waterfall:
A nice place to cool off.
Bryce’s Quarters: Living quarters for Bryce, lead scientist.
Laboratory
Laboratory:
Research lab facilities
Living Quarters: Living quarters for team members
39
Goal Recognition Example
Crystal Island Virtual Environment Map
Living
Quarters
Infirmary
Waterfall
Camp
Entrance
Bryce’s
Quarters
Dining
Hall
Waterfall:
A nice place to cool off.
Bryce’s Quarters: Living quarters for Bryce, lead scientist.
Laboratory
Laboratory:
Research lab facilities
Living Quarters: Living quarters for team members
40
Goal Recognition Example
Crystal Island Virtual Environment Map
Living
Quarters
Infirmary
Waterfall
Camp
Entrance
Bryce’s
Quarters
Dining
Hall
Waterfall:
A nice place to cool off.
Bryce’s Quarters: Living quarters for Bryce, lead scientist.
Laboratory
Laboratory:
Research lab facilities
Living Quarters: Living quarters for team members
41
Goal Recognition Example
Crystal Island Virtual Environment Map
Living
Quarters
Infirmary
Waterfall
Camp
Entrance
Bryce’s
Quarters
Dining
Hall
Waterfall:
A nice place to cool off.
Bryce’s Quarters: Living quarters for Bryce, lead scientist.
Laboratory
Laboratory:
Research lab facilities
Living Quarters: Living quarters for team members
42
Goal Recognition Example
Crystal Island Virtual Environment Map
Living
Quarters
Infirmary
Waterfall
Camp
Entrance
Bryce’s
Quarters
Dining
Hall
Waterfall:
A nice place to cool off.
Bryce’s Quarters: Living quarters for Bryce, lead scientist.
Laboratory
Laboratory:
Research lab facilities
Living Quarters: Living quarters for team members
43
Goal Recognition Example
Crystal Island Virtual Environment Map
Living
Quarters
Infirmary
Waterfall
Camp
Entrance
Bryce’s
Quarters
Dining
Hall
Waterfall:
A nice place to cool off.
Bryce’s Quarters: Living quarters for Bryce, lead scientist.
Laboratory
Laboratory:
Research lab facilities
Living Quarters: Living quarters for team members
44
Goal Recognition Example
Crystal Island Virtual Environment Map
Living
Quarters
Infirmary
Waterfall
Camp
Entrance
Bryce’s
Quarters
Dining
Hall
Waterfall:
A nice place to cool off.
Bryce’s Quarters: Living quarters for Bryce, lead scientist.
Laboratory
Laboratory:
Research lab facilities
Living Quarters: Living quarters for team members
45
Goal Recognition Example
Crystal Island Virtual Environment Map
Living
Quarters
Infirmary
Waterfall
Camp
Entrance
Bryce’s
Quarters
Dining
Hall
Waterfall:
A nice place to cool off.
Bryce’s Quarters: Living quarters for Bryce, lead scientist.
Laboratory
Laboratory:
Research lab facilities
Living Quarters: Living quarters for team members
46
Goal Recognition Example
Crystal Island Virtual Environment Map
Living
Quarters
Infirmary
Waterfall
Camp
Entrance
Bryce’s
Quarters
Dining
Hall
Waterfall:
A nice place to cool off.
Bryce’s Quarters: Living quarters for Bryce, lead scientist.
Laboratory
Laboratory:
Research lab facilities
Living Quarters: Living quarters for team members
47
Goal Recognition Example
Crystal Island Virtual Environment Map
Living
Quarters
Infirmary
Waterfall
Camp
Entrance
Bryce’s
Quarters
Dining
Hall
Waterfall:
A nice place to cool off.
Bryce’s Quarters: Living quarters for Bryce, lead scientist.
Laboratory
Laboratory:
Research lab facilities
Living Quarters: Living quarters for team members
48
Goal Recognition Example
Crystal Island Virtual Environment Map
Living
Quarters
Infirmary
Waterfall
What is the player’s
current goal?
Camp
Entrance
Bryce’s
Quarters
Dining
Hall
Waterfall:
A nice place to cool off.
Bryce’s Quarters: Living quarters for Bryce, lead scientist.
Laboratory
Laboratory:
Research lab facilities
Living Quarters: Living quarters for team members
49
Goal Recognition Example
Crystal Island Virtual Environment Map
Living
Quarters
Infirmary
Waterfall
Camp
Entrance
Bryce’s
Quarters
Dining
Hall
Waterfall:
A nice place to cool off.
Bryce’s Quarters: Living quarters for Bryce, lead scientist.
Laboratory
Laboratory:
Research lab facilities
Living Quarters: Living quarters for team members
50