ILMDA: An Intelligent Agent
that Learns to Deliver Learning
Materials
Leen-Kiat Soh
Department of Computer Science and
Engineering
University of Nebraska
lksoh@cse.unl.edu
What is an Agent?
• An agent is an entity that takes sensory input
from its environment, makes autonomous
decisions, and carries out actions that affect the
environment
– A thermostat is an agent
– A calculator is not an agent
Agent
sensory
input
think!
Environment
output
actions
What is an Intelligent Agent?
• An intelligent agent is one that is capable of flexible
autonomous actions in order to meet its design
objectives, where flexibility means: (Wooldridge and Jennings 1995)
– Reactivity: agents are able to perceive their environment, and
respond in a timely fashion to changes that occur in order to
satisfy their design objectives
– Pro-activeness: agents are able to exhibit goal-directed behavior
by taking the initiative in order to satisfy their design objectives
– Social ability: agents are capable of interacting with other
agents (and possibly humans) in order to satisfy their design
objectives
What is an Intelligent Agent? Cont’d
• Machine Learning in AI says
The acquisition of new knowledge and motor and cognitive skills
and the incorporation of the acquired knowledge and skills in
future system activities, provided that this acquisition and
incorporation is conducted by the system itself and leads to an
improvement in its performance.
• Agents that learn are intelligent
– Individual learning
– Multiagent system learning
• Not all agents are intelligent!
Environment?
• Inaccessible vs. accessible
– Incomplete vs. complete data
(Russell and Norvig 1995)
• Deterministic vs. non-deterministic
– Certainty vs. uncertainty
• Episodic vs. non-episodic
– Each episode is independent or not
• Static vs. dynamic
Agent
– Remain unchanged except by the performance of
actions by the agent?
• Discrete vs. continuous
– “Chess game” vs. “taxi driving”
Why Agents?
• If the system-to-be-built has, during the execution of the
system
Why does a person hire an agent?
–
–
–
–
–
Incomplete data
Uncertainty in the assessment/interaction of its environment
Inter-dependent episodes of events
No full control over the events in the environment
An “open world”, instead of a “closed world”
• In other words, agents are used when you need to build
a system that is adaptive to an uncertain, dynamic, and
at times unexpected environment
– So you can make full use of the autonomous property of an
agent
Education Systems
• Not all computer-aided learning and teaching
systems are agent-based, not all are intelligent
• Systems related to agents focus on two areas:
– Intelligent User Interface
– Tutors
Educational Applications
• Intelligent User Interfaces deliver information to users in a
variety of presentation schemes and interaction schemes
– SAM (Cassell et al. 2000) encourages young children to engage in
storytelling by taking turns playing with a figurine and a toy castle in
SAM’s virtual reality
– GrandChair (Smith 2000) appears to be a young child in a rocking
chair to listen to a grandparents’ family stories
– AIA, PIP, IMP, and Magic Monitor (André and Rist 2001): examples of
controlling agent behavior in intelligent interfaces
– Herman the Bug (in Design-A-Plant), Cosmo the Internet Advisor, and
WhizLow (in a CPU city) (Lester et al. 2002)
Educational Applications
• Intelligent Tutoring Systems provide feedback and support
for student learning
– PACT (Koedinger et al. 1997): algebra, geometry, and computer
languages
– ANDES (Gertner and VanLehn 2000; VanLehn 1996): physics
– SHERLOCK (Lesgold et al. 1992): electronics
– ATLAS (VanLehn et al. 2000)
– AutoTutor (Graesser et al. 2001): introductory course in computer
literacy
– EVELYN Reading Coach and EMILY Reading Coach (Mostow and Aist
2002): Project LISTEN
• Help students read by listening to children read aloud
ILMDA:
Intelligent
Learning
Materials
Delivery
Agent
OVERVIEW
Historical profile,
Real-time behavior
Parametric profile of
student and environment
Retrieval instructions
Profile updates
Statistics updates
ILMDA
Reasoning
student
lectures
Computer Timely delivery
of examples &
& GUI
exercise problems
Examples
Exercise problems
Statistics
database
ILMDA Agent
OVERVIEW 2
login
Quit/Failed Login
New user
New
user
panel
OVERVIEW 2
Successful
Login
Pick
tutorial
panel
Selected tutorial
Tutorial
panel
Quit
Review
Example wanted
Quit
Example
panel
Review
Problem wanted
Qui
t
Problem
panel
METHODOLOGY
• An intelligent agent that
– interacts with its environment (through its GUI),
– makes decisions autonomously using case-based
reasoning
– learns to improve its performance
• A portable design with a Java-based GUI
frontend
• A flexible design with a mySQL database
backend
Case-Based Reasoning
• Case-based reasoning (CBR)
– There is a casebase with numerous cases, each case
dealing with a particular problem with a particular
solution
• A case has a problem, a solution, and an outcome
– The useful part occurs when we encounter a new
problem – how do we find a solution?
• From scratch?
• Or perhaps, use a previous solution?
CBR 2
• How to use a previous solution?
problem
Retrieve
casebase
Retain
Reuse
Revise
confirmed solution
proposed solution
CBR 3
• How to use a previous solution?
Old, but very
similar problem
new problem
Difference?
Changed solution;
Hopefully it works
Adaptation
old, working solution
CBR in ILMDA
• Problem
– Student background, student dynamic activity, profile
of learning materials
• Solution
– Parameters specifying a particular type of examples
or problems to deliver to the students
• Outcome
– Whether the student quits, finishes the entire
learning material, or answers the problem correctly
CBR in ILMDA 2
• Problem Description
– Student Background
• GPA, majors, interests, motivation, self-efficacy, etc.
– Student Dynamic Activity
• Time spent on a tutorial, number of mouse clicks, number of
times going back and forth from the examples to the
tutorial, etc.
– Learning Material Profile
• Tutorials, examples, problems
• Difficulty level, length, number of times viewed, etc.
CBR in ILMDA 3
• Solution Parameters
–
–
–
–
–
–
–
–
Number of times viewed
Difficulty level
Average use time in seconds
Average mouse clicks
Length in characters
Content
Bloom’s category
Scaffolding level (HIGHLIGHT, HINT, REFERENCE,
ELABORATION)
Content
• Example
– An example for the topic on File I/O
Let us say that you are keeping track of your baseball team’s player stats
using a Java program, and you want to be able to save these stats to a file.
Instead of worrying about using high-level File I/O and
(HIGHLIGHT)keeping track of what order things are read
in(/HIGHLIGHT), we can use Object I/O and store things much easier!
(HINT)Why would using Object I/O make storing things much
easier?(/HINT) (HINT)What are the differences between high-level File
I/O and Object I/O?(/HINT) (REFERENCE)If you wonder what the
differences between high-level File I/O and Object I/O are, please check
out the Tutorial page, under the headings of High-Level File I/O and
Object I/O!(/REFERENCE)
Content 2
• Example
– A problem
for topic
File I/O
Difficulty: 0.3
Blooms: Knowledge
Content: Generic
Which of the following must be done in order to use
(HIGHLIGHT)File Input/Output(/HIGHLIGHT) in Java:
(HINT)Think about what you need to use File I/O objects,
what reading from or writing to a file may entail, and what
you need to do to conclude the
process.(HINT)(REFERENCE)To refresh, check out the
Tutorial page, under the File Object heading.(/REFERENCE)
(REFERENCE)(Check out the Example
page.)(/REFERENCE) (ELABORATION)Think about this:
first you need to be able to access IO-related classes and
packages. Then when you set up your program, it is possible
that a file is not there, or the program does not have
permission to read/write, and thus the program must be able to
handle those scenarios. Finally, to prevent data loss, you need
to make sure that the files are closed.(/ELABORATION)
a. Import the java.io libraries
b. Catch or throw Input/Output Exceptions
c. Close the files when we are done using them
CBR in ILMDA 4
• Outcomes
– Level 0: Did the student quit midway through an
example/problem? Did the student fail to answer the
problem correctly?
– Level 1: Do students with exposure to ILMDA
perform better, based on a control-treatment study
on post-tests?
– Level 2: Are agents with machine learning
capabilities more efficient/effective?
– Level 3: Are agents able to provide efficient
“customized” paths for students of different profiles?
Preliminary Results
• Deployment Fall 2004
–
–
–
–
CSCE155 (CS1)
Diverse students (majors, aptitude)
3-hour lectures, 2-hour labs, weekly
Four lab sections
• 20-25 students each
– Used ILMDA in five labs
• File I/O, Event-Driven Programming, Exceptions,
Inheritance/Polymorphism, and Recursion
• Each topic has a tutorial, a set of 3-4 examples, and a set of
20-25 problems
• Students were required to review these learning materials
Preliminary Results 2
• In terms of lab post-tests
– No significant results
Sections
File IO
Event Driven Exceptions Inheritance
Recursion
151
6.89
8.67
8.56
6.44
9.78
152
6.73
8.09
8.90
6.67
8.00
151 + 152
6.80
8.35
8.74
6.56
8.84
153
7.36
8.55
8.73
7.63
8.84
154
7.96
8.18
8.33
7.78
9.42
153 + 154
7.66
8.36
8.55
7.70
9.13
* Averages, maximum 10 points for each post-test
Preliminary Results 3
• Learning vs. non-learning ILMDA
– What does it all mean?
NonLearning Learning
ILMDA
ILMDA
Average number of examples read
2.22
3.05
Average number of problems read
10.95
15.12
66.2%
60.2%
457
276
Average time spent on examples (s)
64
61
Average time spent on problems (s)
74
30
Percentage of problems answered correctly
Average spent on Tutorial (s)
Continued Experiments
• Spring 2005
– 3-hour lectures, 2-hour labs, weekly; five topics
– Three lab sections
• 20-25 students each
– Lab 1: Uses statically sequenced examples and
problems
– Lab 2: Uses non-learning ILMDA
– Lab 2: Uses learning ILMDA
– Dependent variables: post-tests, exams, homework
assignments
• Objective measure: how a student reaches one of the most
difficult problems and answers it correctly
Summary: Ultimate Goal
• Creating an intelligent tutoring system that is able to
learn
–
–
–
–
how to model students correctly
how to evaluate the course materials that it has been given
how to apply appropriate strategies in its delivery
how to integrate all three learned expertise or experience to
tutor students better
• Creating an intelligent tutoring system that does not
assume that the student modeling, the course materials,
and the delivery strategies are always appropriate or
correct
Summary: Current Status
• Prototype in place for two semesters in CS1, with five
CS1 topics
• Learning mechanisms in place for learning how to
deliver materials better
• Basic research work under way for multi-awareness to
determine the “inappropriate component”
• Online GUI tools
–
–
–
–
–
For
For
For
For
For
content entry (completed)
student view statistics query (completed)
instructor view statistics query (completed)
agent view statistics query (partially)
content view inspection (completed)