U.S. Army Research, Development and Engineering Command
Use of Evidence-based
Strategies to Enhance the
Extensibility of Adaptive
Tutoring Technologies
Benjamin Goldberg, Ron Tarr, Dr. Deborah
Billings, Naomi Malone, Keith Brawner, and
Dr. Robert Sottilare
1
Push for Tailored Training
Computer-based tutoring systems (CBTS) have demonstrated
significant promise in tutoring individuals in well-defined
domains, but…
Fifty years of research have been unsuccessful in making
CBTS ubiquitous in military training… Why?
CBTS are expensive to author and are insufficiently
adaptable to support the tailored, self-regulated , individual &
small unit tutoring experiences required to support:
 U.S. Army Learning Model (ALM) for
2015 (TRADOC, 2011)
 U.S. Air Force (AETC, 2008)
 U.S. Navy STEM Grand Challenge (ONR,
2012)
 OSD R&T Vision for PAL
 NATO HFM RTG 237 (Advanced ITS)
 TTCP HUM TP-2 (Training Panel)
Computer-Based Tutoring Systems
(CBTS)
• ITSs apply Artificial Intelligence tools and methods to individualize
instruction
 Based on benefits associated with one-on-one expert tutoring
(2-Sigma Problem; Bloom, 1984)
 Mediates learning by providing feedback when appropriate and
adjusting difficulty levels to maintain desired challenge
 Facilitated by 4 common components
3
Generalized Intelligent
Framework for Tutoring (GIFT)
4
Pedagogical Modeling
• Designed to balance the level of guidance a learner needs with the goal of
maintaining engagement and motivation
5
Macro-Adaptive Strategies
• Organized and sequential set of tactics to be implemented
online
 What to adapt and how to adapt
• Addresses four instructional design areas:
 Selection
 Sequencing
 Synthesizing
 Summarizing
• Based on metrics collected prior to the commencement of
instruction
6
Macro-Adaptation:
Sources of Adaptation
Student Performance/
Achievement Levels
Working Memory Capacity
(WMC)
Knowledge Type
MATERIAL: Task Difficulty/
Complexity
Source of Adaptation
(Individual
Differences & Task
Characteristics)
Learner’s Prior Knowledge/
Expertise
Learner’s Traits/Ability/
Attributes
Learning Styles/Cognitive Styles
IMI Levels
Task Learning Category:
Cognitive, Affective,
Psychomotor, Social
7
“Sources of adaptation” refer to
the factors that prompt, or trigger,
adaptation to occur, namely the
characteristics of learners that
elicit specific instructional tactics to
be implemented.
Macro-Adaptation
Targets of Adaptation
Sequence of Instruction
Presentation of
Information (Graphics,
Animations, etc.)
Target of
Adaptation
(Instructional
Tactics)
Degree of Learner Control
Feedback (Frequency,
Content)
Problem Difficulty
Pace of Instruction
8
Different sources of adaptation
appear to be linked with different
targets of adaptation (as shown)
in the literature.
Based on:
- The instructional elements that
are adapted
- Elements are adapted
according to the specific sources
of adaptation
Home
Tutorial
Instructor
Training Developer
The Instructional Strategies Indicator
(ISI) is a computer-based searchable tool
for selecting individualized instructional
strategies based on the type of
knowledge being learned, the domain in
which the learning occurs, the expertise
level of the learner, and the size of the
group being taught.
Our goal is to inform the selection of instructional strategies in order to improve learning effectiveness and
efficiency across a wide range of domains. Ultimately, the ISI is intended to provide recommendations
regarding how, what, and when to embed instructional components into simulation-based training systems.
To start a search, click on the Strategies tab at the top.
Log in: User name ______________
Password _________________
Application in GIFT:
Model Development
10
Illustrative Example
PRE-TRAINING
[Human]
Instructor Input
PRE-TRAINING
Assessment of
Trainee
Instructional Content
Learning
Objectives
Knowledge
Type
Trainee Characteristics
Declarative
Novice
Conceptual
Journeyman
Procedural
Integrative
GIFT’s Pedagogical
Model
Instructor
Input
Higher Level Instructional
Tactic
Higher Level Instructional
Tactic
Increase “fog” in
Scenario
11
EXAMPLE:
Increase Scenario
Complexity
Goal
Orientation/M
otivation
Self-Efficacy
Expertise
Level
Expert
Extensibility
• The need for Empirical Evaluation of a Generalized
Pedagogical Model to determine its utility across multiple
domains and training platforms
• Goal of Experimentation
 Determine best practices for “Best-in-Class” model to be
incorporated in GIFT
 Determine influence Individual Differences have on
training outcomes
 Determine how instructors and trainers will use
prescribed outputs from model implementation
12
Road-Ahead
• Development of a complementary Micro-Adaptive
Pedagogical Model for use in GIFT
 Used to inform ‘in-situ’ strategies and tactics on a general
level (e.g., provide hint, adjust difficulty, perform
assessment, etc.)
 Requires techniques to monitor reactive states (i.e.,
cognitive and affective) and strategies to mitigate
negative states
• Soldier Centered Army Learning Environment (SCALE)
 Utilize GIFT as SCALE’s pedagogical management engine
 Provide mechanisms of personalized/tailored instruction
with distributable training content
13
QUESTIONS
14