1
Cognitive Efficiency of Animated Pedagogical Agent for Learning Second Language
A Proposal Presented to the
Faculty of the Graduate School
University of Southern California
Sunhee Choi
University of Southern California
to
Dr. Richard E. Clark (Chair)
Dr. Edward Kazlauskas
Dr. Harold F. O’Neil, Jr.
Dr. Robert Rueda
Dr. Nam-kil Kim (Outside Member)
5424 Newcastel Ave. #229
Encino, CA 91316
(213)407-3378
sunheech@usc.edu
2
Table of Contents
ABSTRACT ................................................................................................................................... 3
CHAPTER I: REVIEW OF LITERATURE .............................................................................. 5
Introduction ................................................................................................................................... 5
Learning Second Language Grammar ....................................................................................... 8
Attention and Awareness in Second Language Learning ........................................................ 8
Instructional Methods to Focus Learner Attention ................................................................11
Studies on the Effects of Implicit and Explicit Methods ...................................................... 12
Summary ............................................................................................................................... 18
Cognitive Efficiency of Multimedia Learning .......................................................................... 20
Theoretical Constructs Relevant to Cognitive Efficiency .................................................... 21
Mental Effort Measurement .................................................................................................. 22
Self-Report Measures.................................................................................................... 22
Dual-Task Measures ...................................................................................................... 23
Building Cognitive Efficiency in Multimedia Learning ....................................................... 25
Instructional Strategies to Reduce Cognitive Load .............................................................. 26
Avoiding split attention effects and Utilizing modality effects .................................... 28
Leaving out redundancy effects .................................................................................... 29
Integrating Leaner Prior Knowledge and Expertise...................................................... 31
Summary ............................................................................................................................... 32
Animated Pedagogical Agents .................................................................................................... 34
Benefits of Animated Pedagogical Agents ............................................................................ 35
Motivating Learners ...................................................................................................... 36
Focusing Learner Attention........................................................................................... 41
Summary ............................................................................................................................... 44
Research Hypotheses .................................................................................................................. 46
CHAPTER II: Methodology ...................................................................................................... 53
Target English Grammar for Instruction ................................................................................. 53
Participants .................................................................................................................................. 53
Research Design .......................................................................................................................... 53
Measures ...................................................................................................................................... 54
Data Analysis ............................................................................................................................... 55
References .................................................................................................................................... 59
3
ABSTRACT
An animated pedagogical agent is a lifelike animated character that inhabits a computerbased learning environment, and provides learners with pedagogical assistance such as directing
attention and giving contextualized advice. Recent studies that compare animated pedagogical
agents with alternative media have resulted in ambiguous findings. Clark has claimed that
animated pedagogical agents do not provide instructional benefits unless they deliver essential
instructional methods and that any method can be implemented in a variety of media with equal
learning impact (1983, 1994a, 1994b, 2001, In Press). He further argues that while animated
pedagogical agents are able to present essential instructional methods, other less expensive and
less distracting implementations of instructional methods are equally effective for learning.
The present study explores the use of APA’s as well as other delivery systems in a
multimedia environment in which college level students learn English as a Second Language
(ESL). Experiments will be conducted to assess the effects of two different instructional methods
(metalinguistic rule presentation and visual enhancement) on the acquisition of English relative
clauses which is measured through the comprehension and production tests.
The study also examines the cognitive efficiency of different delivery media including
animated pedagogical agents and a simple flashing arrow used to deliver one of the instructional
methods - visual enhancement. Cognitive efficiency refers to “one medium being more or less
effortful than another, more or less likely to succeed with a particular learner, or interacting more
or less usefully with a particular prior knowledge set” (Cobb, 1997, p. 25), leading to faster
learning, or requiring less conscious effort from learners for processing learning material. The
underlying premise of cognitive efficiency is that a specific medium used to present instruction
may not produce different cognitive outcomes compared to another medium, but it still can have
4
direct impact on cognitive processes.
The main argument of this study is that what causes learning is an instructional method,
not a delivering medium such as animated pedagogical agents or flashing arrows. Therefore, the
study hypothesizes that there will be no significant difference in learner performance on the
target L2 grammar among participants who receive the same instructional methods that are
delivered through different media. In addition, the present study argues that different media
employed to deliver the same instructional method will have significant, different effect on levels
of cognitive efficiency.
The design of the study is a true experimental with pre- and post test involving five
treatment groups and one control group. The treatment groups will be different with regard to
provision of explicit explanations on the target form and the provision of visual enhancement
technique to focus learners’ attention on the target form. The participants will be randomly
assigned in equal proportion to one of the six conditions. There are total five dependent variables
in the present study which include mental effort measures, time measures, interest measures,
attention measures, and performance measures. Descriptive statistics will be used throughout the
study to estimate all measures. Additionally, Multiple Regression and MANOVA will be used to
investigate the relationship among mediating (i.e., learner prior knowledge), process (i.e., time
required to acquire the form, the amount of attention paid to the form), and outcome measures
(i.e., scores for comprehension and production tests).
5
CHAPTER I: REVIEW OF LITERATURE
Introduction
The primary purpose of the present study is to examine the claims that Animated
Pedagogical Agents (APA’s), when used in media-based instructional programs, increase learning
scores over instructional treatments that do not employ APA’s (Atkinson, 2002; Johnson, Ricke,
& Lester, 2000; Lester, Converse, Stone, Kahler, & Barlow, 1997). An APA is a lifelike animated
character that inhabits a computer-based learning environment, and provides learners with
pedagogical assistance such as directing attention and giving advice about learning strategies.
Recent studies that compare APA’s with alternative treatments (Atkinson, 2002; Craig, Gholson,
& Driscoll, 2002; Moreno, Mayer, Spires, & Lester, 2001) have resulted in ambiguous findings.
Clark has claimed that APA’s (and other media and media attribute treatments) are not
instrumental in learning unless they provide essential instructional methods and that any method
can be implemented in a variety of ways with equal learning impact (1983, 1994a, 1994b, 2001,
In Press). He further argues that while APA’s are able to present essential instructional methods
such as instructional plans, examples, and feedback, other less expensive and less distracting
implementations of instructional methods are equally effective for learning (i.e., using simple
arrows or color coding of key points rather than having an APA “point” to text or parts of
instructional graphics). Therefore, Clark concludes, it is the instructional method used, not the
specific medium or audio-visual agent used to deliver the method that leads to learning gains.
Nevertheless, this argument is not settled. On the one hand, with the wide availability of
multimedia and information technology, an ever increasing number of educational software
programs are promoting the inclusion of multimedia elements in instruction as a panacea to
6
learning problems (Kimmel & Deek, 1996; Lowe, 2002). And animated pedagogical agents are
simply the latest iteration of recent technological advances in user interface and autonomous
software agents that are being developed to aid instruction. On the other hand, a number of
researchers and educational economists are concerned that unnecessarily expensive instructional
tools are being proposed to solve critical learning and educational access problems when less
expensive options would have either equal or greater impact on learning (Erickson, 1997; Levin
& McEwan, 2001).
The present study will explore the use of APA’s as well as other delivery systems in a
multimedia learning environment in which college level students learn English as a Second
Language (ESL). Experiments will be conducted to assess the effects of two different
instructional methods (metalinguistic rule presentation and visual enhancement) on the
acquisition of English relative clauses which is measured through comprehension and production
tests. In order to address the issue, the study will adopt a 2 (metalinguistic rule explanation: rule
presentation, no rule presentation) by 3 (visual enhancement: animated pedagogical agent,
flashing arrow, no visual enhancement) true experimental design. Other dependent variables will
also be measured to estimate the relative effectiveness and efficiency of different methods and
media used in the study, which include the mental effort, learning time, learner interest, and
attention.
In addition, the study will examine the cognitive efficiency of different delivery media
including animated pedagogical agents and a simple flashing arrow used to deliver one of the
instructional methods - visual enhancement. According to Cobb (1997), who proposed to include
‘Cognitive Efficiency’ as a variable in media studies, cognitive efficiency refers to “one medium
being more or less effortful than another, more or less likely to succeed with a particular learner,
7
or interacting more or less usefully with a particular prior knowledge set” (p. 25), leading to
faster learning, or requiring less conscious effort from learner for processing learning materials
(Cobb, 1997; Clark, 1998).
The underlying principle of cognitive efficiency is that a specific medium through which
instruction is presented to learners may not produce different cognitive outcomes compared to
another, such as a superior mental representation, but it still can have direct impact on cognitive
processes with different levels of cognitive efficiency. In other words, different media in which
an instruction is delivered might end up with the same cognitive product, but they can determine
how different learners with different prior knowledge process the information presented to them.
Despite its potential for multimedia- and computer-assisted learning, cognitive efficiency is still
in the early stage of development and needs to have sound theoretical frameworks and empirical
evidence to support its hypothesis.
To lay out the foundation of the present study, the following sections will review
relevant theories and research findings and investigate how to build cognitive efficiency of
multimedia learning using an animated pedagogical agent and an alternative flashing arrow. First,
the review will look at the factors involved in learning second language (L2) grammar, which
will be the learning task as well as the dependent variable of the present study. In particular, it
will focus on instructional methods that have been found to enhance the acquisition of L2
grammar. Secondly, the review will look at the factors relevant to cognitive efficiency and
examine the ways to improve cognitive efficiency in multimedia learning environment. Finally,
the review will summarize what and how studies have been conducted in the field of animated
pedagogical agent and closely examine APA’s from a cognitive efficiency perspective, and then
outline the challenges facing the field including the need for a sound study design to clarify to
8
what factors learning outcomes can be attributed.
Learning Second Language Grammar
The status of grammar instruction in second and foreign language (L2) education has
changed throughout history as theories about the nature of language and language acquisition
have changed (Richards & Rodgers, 1998). Over the last two decades, in particular, the question
of the role of grammar instruction in L2 acquisition, whether it facilitates L2 acquisition or not,
has been a major topic of debate, and thereby has produced a considerable number of theories
and studies (Norris & Ortega, 2000). Yet, the focus of L2 instruction research has gradually
moved from whether L2 instruction is necessary to what instructional methods are more effective
for L2 grammar acquisition (Norris & Ortega, 2000), since Long (1983) concluded that L2
grammar instruction indeed makes difference in learner performance after comparing the learner
achievement of explicit instruction with that of natural exposure or combination of two. Over the
last two decades, a number of quasi-experimental and experimental studies have been conducted
to examine the effectiveness of various instructional treatments. Despite the variety of the
instructional methods, the basic premise of these methods is the same: an instructional treatment
should attract learners’ focal attention to the target L2 form, another common term for grammar
in the field of Second Language Acquisition (SLA), within a meaningful context so that the form
is more likely to be noticed, processed, and acquired (Schmidt, 1995; Spada, 1997). The role of
attention in second language learning and measurement issues are discussed in the next section
and several instructional methods to draw learner attention in the following section.
Attention and Awareness in Second Language Learning
The role of attention and awareness in learning has been extensively studied both in
9
psychology and second language acquisition (Curran & Keele, 1993; Dulany, 1991; Reber, 1976,
1989, 1993; Schmidt, 1995; Tomlin & Villa, 1994). The general consensus in SLA is that
attention to a certain linguistic feature is required for learning to take place. However, it is still
controversial how much and what type of subjective awareness of an L2 form is necessary for
learning to occur (Izumi, 2002). There are three major positions regarding this question. First,
Schmidt (1990, 1995, 2001), based on his Noticing Hypothesis, claims that “what learners notice
in input becomes intake for learning” (1995, p. 20), Here, noticing is referred to as cognitive
registration of stimuli (e.g., meaning or form of L2 input) in conscious awareness followed by
storage in long-term memory, and intake as the portion of input which has been perceived and
processed by learners. From Schmidt’s point of view, therefore, learning cannot take place
without learners’ subjective awareness at the level of noticing.
On the contrary, Tomlin and Villa (1994) proposed a theoretical model of attention in
which attention is divided into three interrelated processes; alertness – “general readiness to deal
with incoming stimuli or data” (p. 190), orientation – “the specific aligning of attention on a
stimulus” (p. 191), and detection – “the cognitive registration of sensory stimuli” (p. 192).
Drawing from this model, they argue that conscious awareness is not necessary for learning even
though it might facilitate learning, and what is crucial for learning is the detection which does
not require learners’ conscious awareness.
The third position comes from Robinson (1995) who posits himself in the middle of the
previous two positions. He contends that learning might happen when a learner detects the form
without focal attention or subjective awareness, but the amount of learning would be very limited.
Rather, in order to store a stimulus into memory one has to detect and notice the target form,
which then should be rehearsed in short-term memory. Despite this controversy, it is now widely
10
agreed that conscious, focal attention is necessary for learning to take place and a higher level of
subjective awareness or noticing is correlated with better learning (Rosa & O'Neill, 1999).
Therefore, the present study will measure learner attention by examine whether learners have
noticed the form. Then, it will be hypothesized that participants’ conscious awareness of the
target L2 form will have significant effect on the level of learner performance measured by
comprehension and production tests on the target grammar.
There is another issue that has been subject to debate in the field - the ways to measure
the amount of learner attention and awareness. Attention measurement is constructed mostly
based on Schmidt’s operational definition of noticing (Schmidt, 1995) - the availability for
verbalizing what one has experienced during or right after exposure to the input. Accordingly,
many measuring methods employ learner self-report or questionnaires that are believed to
capture what learners have experienced during the exposure to a grammatical form. Yet, it should
be noted that some experiences are not easy to report or verbalize, and hence, the lack of selfreport does not necessarily mean the lack of awareness (Schmidt, 1995). Leow (1998) also
pointed out that the indirect post-experiment methods may obtain only indirect evidence of
learners’ attention to the form. Moreover, retrospective self-report measures are restricted to a
certain degree in that that learners might have limited ability to retain in their memory what they
have experienced, and they could report what they have inferred instead of what they have
actually experienced (Rosa & O’Neill, 1999).
Recently, researchers have started to use more direct measuring methods in order to
capture what is really happening when learners attend to forms and process them for further
learning. For instance, the think-aloud protocol, through which learners are required to verbalize
whatever comes to their mind during learning process, is one of the popular direct methods used
11
in the field of SLA. Despite its popularity in the field, the use of the think-aloud protocol and its
validity have also been questioned by some researchers. They criticize that the think aloud
protocol could be posed as a secondary task on learners who have to simultaneously process the
input (Izumi, 2002). In addition, there is a possibility that the protocol could change the very
nature of the process under investigation by prompting learners to process the input in a more
systematic way than they would do without thinking aloud (Rosa & O’Neill, 1999).
Instructional Methods to Focus Learner Attention
The present study hypothesizes that different instructional methods will have different
cognitive effects on learning L2 grammar. Thus, it is necessary to review instructional methods
for drawing learners’ attention to a linguistic target and their effectiveness in enhancing learner
performance in both comprehending and using the target form. Several instructional methods
have been developed and studied including textual or visual enhancement, metalinguistic
explanation of the linguistic form (i.e., syntax or morphology) before or after exposure to input,
and frequent use of the target form in input (e.g., input flood). These methods can be categorized
depending upon the degree of their explicitness in requiring learners to focus on the form. An
explicit instruction is centered on providing metalinguistic rule explanation, and/or tasks that
explicitly require learners to pay attention to specific forms, whereas an implicit instruction
utilizes unobtrusive techniques and/or tasks (e.g., visual manipulation of forms) to lead learners
to notice forms. Implicit methods intend to lead learners to focus not only on the target form, but
also on the meaning of input (DeKeyser, 1995).
Different instructional treatments have a different selection and arrangement of the
methods listed above. Among several methods, however, the metalinguistic rule explanation and
visual enhancement of the target form have been most discussed in the field and their relative
12
effectiveness have been investigated, in particular, when they are matched up with other
moderating factors including learners’ L2 prior knowledge or proficiency level, learner
characteristics (i.e., language aptitude), and the target language forms (e.g., simple vs. complex
linguistic features) (Norris & Ortega, 2000). These two methods are also the main instructional
methods employed in the present study that will be delivered by using different forms of
multimedia. Furthermore, their effectiveness will be studied with relation to learners’ prior
knowledge of the target form to get a more complete understanding of their effects on learning.
The following section, hence, will review a body of literature regarding the effectiveness of the
explicit rule presentation and visual manipulation, keeping in mind that there could be other
factors that could affect learners’ interaction with instructional methods and subsequent results.
Studies on the Effects of Implicit and Explicit Methods
Visual input enhancement (i.e., the targeted linguistic features are italicized, bold, or
capitalized for perceptual salience), an implicit and unobtrusive attention drawing technique, is
hypothesized to help learners to focus on a specific grammatical structure contained in written
text by making the form perceptually salient. The fundamental idea behind visual enhancement is
that learners’ attention will be drawn to the highlighted grammatical structure in input, and then
the attended form will be learned because attention transforms input into intake (Izumi, 2002). It
is also based on the notion that explicitly drawing learners’ attention to linguistic forms interferes
with communication and meaning-making process. In contrast, explicit methods assume that
simply exposing learners to the target form is not sufficient to enable learners to learn most of the
L2 grammar and that this lack should be compensated for by explicitly explaining about the
target form (DeKeyser, 1998). The contents of the explicit instruction prime learners for specific
grammatical features so that they pay longer and deeper attention to the target forms and form
13
conscious, accurate knowledge of the forms.
Recently, a number of laboratory and classroom studies have revealed that learners who
are explicitly exposed to target forms and who are given explicit rule explanations outperformed
those who are implicitly directed to forms (Alanen, 1995; DeKeyser, 1995; Ellis, 1994; Robinson,
1996, 1997a, b). On the contrary, the research on implicit methods including the visual
enhancement produced mixed results (Jourdenais, Ota, Stauffer, Boyson & Doughty, 1995). The
mixed results could be due to a number of other independent and mediating factors involved in
studies, such as the length of treatment, additional instructional methods implemented besides
textual input enhancement, attention measured, and learners’ proficiency levels.
For instance, Alanen (1995) found positive effects for metalinguistic rule presentation,
but mixed results of visual enhancement on the acquisition of semi-artificial Finnish locative
suffixes and consonant gradation. In the study, participants were asked to read a text with the
target structure embedded in it after assigned to one control and three treatment groups: (a) the
Control group only read two descriptive passages with a picture and glossary; (b) the Enhance
group received the same text with the target structures printed in italics; (c) the Rule group was
given explicit explanation about the use of the target structures before reading the text; and (d)
the Rule & Enhance group received the explicit rule explanation and the typographically
enhanced text. The first two groups were also referred to as meaning-based groups whereas the
latter two groups were referred to as rule-based groups. Participants were required to think aloud
while reading the text to find out what features of the input they paid attention to, and whether
they had noticed the form.
As hypothesized, the results of various assessments (i.e., think aloud protocol,
grammaticality judgment, sentence completion, comprehension, word translation, and rule
14
statement) revealed that the rule-based groups, the Rule and Rule & Enhance groups,
outperformed the meaning-based groups, the control and Enhance groups, in learning both
locative suffixes and consonant gradation. Interestingly, however, the data analysis did not
support the initial hypotheses that visual input enhancement would have positive impact on the
learning of the target structure. The students in the visual enhancement groups noticed the italics
in the text, but it appeared that not many of them thought about the reasons. As a consequence,
no significant difference was found between the Control and the Enhance group as well as
between the Rule and the Rule & Enhance group. Based on these findings, Alanen concluded
that it was the explicit rule explanation that made differences in learning. With regard to the
failure of the visual input enhancement, Alanen attributed it to the low degree of perceptual
saliency of italics.
The study by Jourdenais et al. (1995) is one of the few studies which found that textually
modified input facilitated the noticing of and the subsequent production of grammatical forms.
Participants, 14 college students learning Spanish, were first assigned to either visual
enhancement or comparison groups, and then given a sample text written in Spanish as a
stimulus for a writing task. However, only the enhancement group received a textually enhanced
sample text in which the target forms, Spanish preterit and imperfect, were manipulated (e.g.,
shadowed, underlined, bolded, etc.). After reading the sample text, the participants were told to
narrate a series of pictures in writing. They were also told to think aloud as they wrote. The
think-aloud protocol was employed to investigate a learner’s concurrent cognitive processing of
the forms and to measure the degree and the nature of noticing. In addition, the students’ written
products were analyzed to evaluate the learners’ use of the target forms.
The think-aloud protocol data revealed that the students in the visual enhancement group
15
made significantly more explicit mentions of the forms than those in the comparison group,
which Jourdenais and colleagues interpreted as that the enhancement group had been better
primed for processing of the target forms due to the textual enhancement. The written
productions also showed that the enhancement group used the forms more often than the
comparison group although the difference in accuracy did not significantly differ. The results,
however, should be interpreted with caution because learners were already familiar with the
forms, and had different proficiency levels in Spanish reading and writing skills. As the
researchers mentioned, it was not clear what role these individual differences played in cognitive
processing as well as outcomes.
An L2 instruction may be composed of more than one method to effectively draw
otherwise elusive learner attention to form. Ellis (1994) insists that the combination of implicit
and explicit method works better than the use of only one method, either implicit or explicit,
since learners need not only to learn the form, but also to integrate the form and the meaning to
successfully complete language acquisition process. White (1998) also suggests that the
typographically enhanced input does not provide enough information about the use of forms, and
therefore, in addition to visually enhancing input it might be more helpful to provide explicit rule
explanation or different types of visual enhancement such as arrows or color-coding which could
clarify the relationship among pertinent elements. She also maintains that individual differences
and some external factors, such as regular classroom activities, should be considered when
studying the effect of instructional methods.
Then, there arises a question of what it means that visual manipulation of text was not
perceptually salient enough. Based on the two studies discussed above, it appears that visual
enhancement is not qualitatively good enough to trigger further cognitive processing beyond
16
mere noticing of forms for acquisition to take place. Consequently, it should be sought out then
what other instructional methods could be used with or without the visual input enhancement
method to promote learners’ cognitive processing beyond mere noticing of forms. Izumi (2002)
partially addressed the issue by investigating the facilitative effects of output practice in addition
to visual enhancement on drawing learners’ attention and consequent learning of grammatical
forms in a controlled experimental study.
The total of 61 adult ESL (English as a Second Language) learners were randomly
divided into different treatment groups, each of which was exposed twice to the target form,
English relative clauses, presented in reading text through different combination of output
practice and visual input enhancement. The output practice was assumed to facilitate learners’
noticing of forms by inducing them to realize problems in their production, which gives learners
heightened awareness of the target form provided in subsequent input. Note-taking of formrelated words was used to measure learner awareness at the level of noticing while learners were
reading texts. Yet, it was also assured that the absence of note-taking did not mean the absence of
noticing of the form. The instruction given for note-taking was different for the output groups
and the input enhancement groups; while the output groups were asked to take notes of any
words which they thought important to the following text reconstruction tasks, the enhancement
groups were asked to take notes for the following reading comprehension tasks. Acquisition of
the forms was measured through various production (e.g., sentence combination, pictured-cured
sentence completion) and reception (e.g., interpretation, grammaticality judgment) tests.
Unlike Alanen (1995), who found positive correlation between the noticing and the
learning of forms in some groups, Izumi did not find such relationship for either output practice
or visual enhancement groups. To be detailed, the visual enhancement had a significant effect on
17
the number of noticing, but failed to produce any measurable gains in learning of the target forms.
On the contrary, the output practice caused great improvements on learning of the forms despite
it did not create any evident impact on noticing of the forms. Izumi explained that the failure of
visual enhancement to facilitate learning was because noticing was not automatically led to
further cognitive processing which might be necessary for acquisition. It was also argued that
what is really important for acquisition of grammatical structure is not the quantity of attention,
but the quality of attention, that is, the level and type of attention and processing. A shallow level
of attention, such as maintaining continued attention to certain forms at one level without
shifting to a deeper processing level, is not enough to prompt learners to learn the target forms.
In this study, the visual input enhancement was claimed to cause only a shallow level of attention
while the output practice led the learners to go beyond the superficial level and acquire the form.
Izumi also called attention to the notion of integrative processing and its relevance to
SLA in that it emphasizes not only learners’ attention to individual elements, but also their
understanding of the relationship among the elements. And it is the latter which was missing in
the visual input enhancement, but present in the output practice. However, Izumi did not provide
any statistical evidence nor explain how the output practice encouraged learners to connect all
the related elements and conceive them as a coherent structure, the English relative clauses. In
addition, it is not clear why the output practice group did not notice the form, but was still able to
learn it. It could be argued that the output practice group’s note-taking for sentence
reconstruction caused qualitatively different cognitive processing than the note-taking for
reading comprehension since as Izumi himself argued that production requirements could lead
learners to process language differently than the simple comprehension requirements.
18
Summary
The current section was devoted to the review of experimental and quasi-experimental
studies with regard to learning L2 grammar. In particular, different instructional methods (e.g.,
visual input enhancement, metalinguistic rule presentation, output practice) to draw learners’
attention to the forms were examined in terms of their effectiveness and limitations as well. Most
of the instructional methods discussed above are based on the basic premise that the amount of
attention to form is positively correlated with the levels of learning the form. Interestingly,
however, the textual manipulation of forms failed to show greater impact on learner performance
despite its posited instructional benefits for drawing learners’ focal attention to forms. On the
other hand, other explicit methods, such as explicit explanation of rules and production practice,
produced measurable gains in terms of noticing and learning the target forms. In many studies,
however, learners who received combination of more than one method (e.g., visual enhancement
and output practice) outperformed other groups who received only either type of method, which
necessitates further research on what other combinations are available and which combination
works best in a specific context.
There are some other issues to be considered. First, focusing on only one instructional
factor, that is, how to draw learners’ attention, the studies did not give much consideration to
other mediating variables involved in L2 learning such as learner prior knowledge. Second, few
studies examined the effects of the learning task itself and the direction on what to do with the
task to induce learners to attend to and to learn the form, despite previous research has revealed
they might have certain impact on them (Rosa & O'Neill, 1999; Williams, 1999). According to
Rosa and O’Neill (1999), their subjects benefited from the problem-solving jigsaw puzzle task
and the type of directions accompanying the task in noticing and processing the form. In
19
particular, they claimed that regardless of the types of treatment group to which they were
assigned, all the participants improved significantly from the pretest to the posttests due to the
properties of the puzzle task (task essentialness and immediate feedback).
Finally, there is a need to develop diverse ways to deliver visual input enhancement.
Previous studies simply highlighted the target form and its components using different fonts and
other textual manipulation (i.e., shadowing, underlining, and italicizing) that did not possibly
give too much information about the relationship among the components. As White (1998)
pointed out, more elaborated methods to indicate the relationship, such as arrows or color-coding,
could help learners to understand the reasons of highlighting the target elements and their
relationships.
Based on the findings and limitations presented in this section, the present study will
examine the effect of a combination of instructional methods – visual enhancement plus
metalinguistic rule presentation as well as the effect of each alone. In particular, it will employ
more elaborated multimedia techniques (i.e., color coding, aural explanation instead of text,
electronic flashing arrow, an animated pedagogical agent) to deliver these methods in a
computer-assisted learning environment. Given the recent trend in the field of L2 instruction that
instructions are increasingly delivered through computers and multimedia, it would be worth
investigating what other multimedia delivery methods are available and which ones are useful
for drawing learners’ attention and facilitating L2 grammar learning. Furthermore, the present
study will conduct a study confirming to methodological standards by taking into consideration
learner prior knowledge and controlling instructional tasks as well as directions for carrying out
them.
20
Cognitive Efficiency of Multimedia Learning
Media researchers now agree to a certain extent that questioning about the effect of one
medium over another on cognitive products is a circular and fruitless argument, and it is time to
move our focus on media impact on learners’ cognitive process and on how to utilize media to
improve the process {Clark, 1998; Mayer, 1997, 2001}. One of the media researchers who are
concerned with how something is learned and how media can help this process is Tom Cobb who
proposed to include ‘Cognitive Efficiency’ as a variable in media studies (Cobb, 1997). As
mentioned above, cognitive efficiency refers to one medium requiring more or less mental effort
or time over another medium (Cobb, 1997; Clark, 1998).
Under the proposition of cognitive efficiency, there are two independent, but interactive
variables; the mode of presenting an instruction and individual or group differences in terms of
their prior knowledge and experience that would influence whether they process the instruction
faster and/or easier (Clark, 1998). Therefore, researchers may ask questions like how many hours
are required for a particular learner to learn a unit of learning material with one particular
medium vs. another medium or how much mental effort a learner invested in processing
information when different media applied. Hence, cognitive efficiency has varying degrees of
relevance to media selection.
Yet, cognitive efficiency is not about choosing the unique and best medium for
presenting an instruction. Rather, it is more about choosing the most cognitively efficient
medium among the alternatives for a given task and given learners. This section reviews
theoretical frameworks and empirical findings associated cognitive efficiency in multimedia
learning. It further looks at instructional and design principles to build cognitive efficiency in
21
multimedia learning environment. These will be the basis for a discussion in the next section
about animated pedagogical agents (APA’s) since an APA is one of the multimedia delivery
methods used in this study.
Theoretical Constructs Relevant to Cognitive Efficiency
As mentioned above, cognitive efficiency has not yet attracted theoretical or empirical
attention from both researchers and educators, in spite of its possibility to move forward the field
of media research. Consequently, it has not been able to build up theoretical support for
conducting empirical studies in order to confirm its promises, which necessitates for the present
study to examine relevant ideas and constructs from existing media studies. Among various
theoretical constructs found in media research, the concept of mental effort is closely related to
cognitive efficiency. Mental effort is typically defined as “the number of non-automatic
elaborations applied to processing a unit of material” (Salomon, 1983, p.44), or “the amount of
actual effort invested in the lesson” which includes “cognitive activities such as perceptual
processing, searching memory for appropriate schemata, and elaborating on the content”
(Cennamo, 1993, p.15).
There are a few concepts similar to mental effort, although there is a lack of agreement
among researchers in defining this type of mental construct (Flad, 2002). For example, Posner
and Snyder (1975) proposed the term ‘conscious attention’ compared to ‘automated attention’ to
indicate a mental process put into operation when making overt responses, retrieving information
from memory, and developing a hypothesis. On the contrary, Shiffrin and Schneider (1977) used
the term ‘conscious attention’ to describe the level of cognitive involvement beyond the
automated processing. There is another construct called ‘cognitive load’ which has been
extensively studied by John Sweller and his colleagues (Sweller, 1999; Sweller & Chandler,
22
1994, Sweller, Cooper, Tierney, & Cooper, 1990; Sweller, van Marrienboer, & Paas, 1998).
Cognitive load refers to the total amount of mental activity imposed on working memory at one
time. What is common among all these concepts is that they are expended only in processing
declarative knowledge (e.g., knowing concepts, processes, and principles), which requires
learners’ conscious attention and processing, rather than highly automated (requiring little no
conscious processing) procedural knowledge (e.g., knowing how to do something).
Mental Effort Measurement
Given its importance in cognitive efficiency, mental effort should be considered and
incorporated into designing multimedia learning. Then, measuring the exact amount of mental
effort invested by learners becomes the first step to take in helping learners learn more efficiently
with media. However, the construct of mental effort is hard to measure directly (Clark, 1999b)
since it is not directly observable, and as a consequence, there have been several methods and
techniques developed to indirectly assess the amount of mental effort. They could be grouped
into three main categories according to what kind of processing they are measuring: self-report
opinion measures, dual-task measures, and physiological measures. Each method corresponds to
the areas of introspection, information processing and neural processing respectively (Cennamo,
1992).
Self-Report Measures
Mostly in a self-report format, subjective opinion measures assume that investing mental
effort is an intentional and non-automatic process, and readily accessible by individuals
(Beentjes, 1989). It also presumes that individuals are relatively capable of reporting the mental
effort which they put in processing learning materials (Salomon, 1984). Opinion measures
include Inventory of Learning Processes (ILP) scales designed to measure individual differences
23
in the level of processing, Amount of Invested Mental Effort (AIME) questionnaire developed by
Gabriel Salomon (1983), and several scales used to assess mental workload imposed in operating
a flight simulator such as the Cooper-Harper aircraft handing rating scales and Workload
Compensation Interference/Technical Effectiveness scales (Casali, Wierwille, & Cordes, 1983).
All of these measures have a reputation of being most efficient to use and having high
reliability (Dweck, 1989). Among them, the AIME questionnaire has been frequently used in the
studies examining differential effect of media on learners’ mental effort investment (Beentjes,
1989; Salomon, 1983; Salomon, 1984; Salomon & Leigh, 1984). Typically, the AIME
instruments ask subjects to answer a set of questions on a 4 point Likert-type scale about how
much effort they think they have invested or how much they concentrated in processing a
particular unit of material (Salomon, 1983). Recently several researchers have questioned the
validity of self-report measures of mental effort and have called for the use of more precise and
multiple methods (Beentjes, 1989; Gimino, 2000).
Dual-Task Measures
Dual-task measures encompass a variety of methods that require a subject to work on a
primary and a secondary task simultaneously. Among the various secondary tasks, finger tapping,
digit shadowing, and memory-scanning tasks are used most. The dual-task method is based on
the notion that one’s cognitive capacity is limited, and when much of the cognitive capacity is
used by a conscious, non-automated primary task, there will be less cognitive capacity available
for the secondary task. The amount of invested mental effort is computed based on the difference
between the baseline performance of the secondary task and the performance in the experimental
condition (Cennamo, 1992) where the primary and secondary tasks are performed at the same
time. The baseline performance of the secondary task is the reaction time to perform the
24
secondary task alone and it is measured before the primary task is presented. The increase in
reaction time to the secondary task in the experimental condition indicates the amount of
conscious effort allocated to the primary task. Several studies that applied various dual-task
measures have found these measures sensitive to differences in cognitive task demands, that is,
different metal effort requirement of different tasks (Gimino, 2000).
When using dual-task methods, it is important for researchers to decide at which point in
the experiment learners are asked to respond to a secondary stimulus. That is because there is
individual difference in the speed of processing the primary task (Gimino, 2000). For instance,
some learners might face the secondary task when they are closer to the solution, which imposes
high cognitive load and causes longer reaction time. On the contrary, learners who are
interrupted by the secondary task closer to the beginning do not need to spend as much mental
effort. In addition, structure and complexity of learning material could influence mental effort
expenditure. That is, although the overall content of the lesson is easy, some parts of the content
may not be so. If learners have to respond to the secondary task while the presented material is
complex, the level of mental effort would be shown high. However, that does not mean the
material requires a great amount of mental effort (Cennamo, 1996).
Physiological Measures
Researchers have been using physiological measures to measure psychomotor,
perceptual, communication and cognitive task demands. Physiological measures assume that
there are physiological reactions in subjects to increases in mental effort spending while the
subjects are performing a task. The difference between a baseline measurement and a
measurement taken in the experimental condition in which a leaner is working on an assigned
task indicates the amount of mental effort. The physiological measures include a range of
25
techniques such as monitoring heart rate, blood pressure, or respiration variability, using
electroencephalogram (EEG), and recording the number of eye blinks per minute (Wierwille,
Rahimi, & Casali, 1985). Although physiological measures are extensively used in the fields like
human factors engineering, they are not very practical to use in classrooms due to high cost
equipments (Gimino, 2000).
Each measurement technique discussed above seizes different aspects of the construct of
mental effort (Fisher & Ford, 1998), which requires researchers and educators to carefully
inspect the purpose of their study and to decide what aspects of mental effort they plan to
measure. Furthermore, research has shown that the sensitivity of different measures depend on
the type of task placed on learners (Casali et al., 1983; Wierwille et al., 1985). Therefore, it is
necessary for educators and researchers to take into consideration the type of learning tasks they
will use when designing a study in order to obtain more accurate results.
Building Cognitive Efficiency in Multimedia Learning
Cognitive load refers to the total amount of mental activity imposed on working memory
at one time (Cooper, 1998), and one of the theories that is concerned with cognitive load is John
Sweller and his colleagues’ ‘Cognitive Load Theory’ (Sweller, 1994; Sweller & Chandler, 1994;
Sweller et al., 1998). A fundamental idea of the cognitive load theory is that working memory is
limited in terms of the amount of information it can hold and process at a time, and thus, it
should not be overloaded with too much information at once (Chandler & Sweller, 1991;
Kalyuga, Chandler, & Sweller, 1998; Sweller & Chandler, 1994; Sweller et al., 1990; van
Gerven, Paas, & Schmidt, 2000). Another important idea of the cognitive load theory is that
working memory consists of partly independent auditory and visual buffers (Baddeley, 1992),
and each buffer processes different forms of information accordingly. With separate working
26
memory buffers being utilized to process information presented in auditory and visual modes, it
is hypothesized that the capacity of working memory will be increased. Consequently, the
amount of information that can be processed for the fixed amount of time will be increased,
which will in turn result in improved efficiency of cognitive processing.
Given the definition of cognitive load and its conceptual similarity to mental effort, it
can be inferred that by reducing the amount of cognitive load placed on working memory, it is
possible to impose less cognitive demand and to request less mental effort from learners, and
eventually improves the efficiency of cognitive processing. Then, how can we reduce the amount
of cognitive load? Cognitive load consists of intrinsic and extraneous loads (Chandler & Sweller,
1991). The former is solely determined by the nature of learning material itself, such as task
complexity or high interactivity of the elements of the learning material, meaning that each
element cannot be learned without reference to other elements. On the other hand, the extraneous
component of cognitive load results from the instructional format or strategy used to deliver the
instruction. The problem with extraneous cognitive loads is that they are irrelevant to learning,
thus not leading to schema acquisition and automation, two major products of human learning
(Sweller & Chandler, 1994). Since it is not possible to change the intrinsic nature of instruction
or to lower the levels of intrinsic element interactivity, it is best to keep extraneous cognitive
load minimum so that the total amount of cognitive load imposed by a task falls within the
limited mental resources of working memory (Chandler & Sweller, 1991).
Instructional Strategies to Reduce Cognitive Load
There are several instructional strategies that can be used to reduce the level of
extraneous cognitive load (Sweller, 1999) and consequently the amount of mental effort: (a)
using goal-free problems that do not allow learners to employ means-ends strategies and
27
backward reasoning. With a typical goal-fee problem unlike a goal-specific problem, a learner
does not have to keep main goals and sub goals of the problem in working memory, which in
turn saves working memory resources; (b) using worked examples in which the problems
accompanied by their worked-out solutions. Worked examples do not force learners to use
means-ends strategy which demands a considerable amount of working memory capacity; (c)
avoiding split-attention effect which results from presenting mutually referring information
separately. For example, when presented with mutually referring information separately, a
learner has to split his or her attention and again mentally integrate them, which results in
imposing a high level of extraneous cognitive load. Instead, the information should be presented
in a physically integrated format; (d) distributing information over different modalities by which
two different modal processors, a visuo-spatial sketchpad and a phonological loop, are utilized
simultaneously in working memory; and (e) avoiding providing redundant information through
different media.
Among these five instructional strategies, the last three have specific relevance to
multimedia and cognitive efficiency, and thus to the present study. Several studies have been
conducted to test if these three presentation modes have significant impact on cognitive
processes and learning outcomes (Jeung, Chandler, & Sweller, 1997; Mayer, Moreno, Boire, &
Vagge, 1999; Moreno & Mayer, 2000a, b; Mousavi, Low & Sweller, 1995; Sweller & Chandler,
1994). Yet, these studies mainly focused on measuring learners’ cognitive products or final
performance, not on their learning processes. So instead of asking questions like ‘how long did it
take to learn the materials presented through a specific mode?’ or ‘Was it easier to learn with one
specific medium over another?’, these studies are more interested to know about whether
learners perform better in a number of performance tests (e.g., knowledge of simple facts,
28
knowledge transfer) after interaction with a certain presentation mode. This lack of studies on
learning process emphasizes the importance of the present study because it will investigate
relative effect and efficiency of different media on learning process as well as final performance
when learning a second language.
Avoiding split attention effects and Utilizing modality effects
The split attention effect occurs when a lesson consists of two or more different sources
of information that are incomprehensible until they are mentally integrated by a learner (Mousavi
et al., 1995), such as a separately presented geometric diagram and accompanying sentences that
are spreading across a page or computer screen. In this situation, learners have to find referential
connections between corresponding aspects of the diagram and sentences, since they cannot be
understood in isolation. This extra mental integration process requires cognitive resource, and as
a result, there might not be enough resources left necessary to achieve more essential learning
objectives, such as schema acquisition. Several research have shown that by physically
integrating multiple sources of learning material (e.g., placing explanations in appropriate places
near a diagram), it is possible to avoid split attention and to prevent learners from investing
mental effort in unnecessary processes (Chandler & Sweller, 1991; Mayer & Moreno, 1998;
Mayer, Heiser, & Lonn, 2001, Experiment 1 & 2; Sweller et al., 1990; Sweller & Chandler,
1994; Ward & Sweller, 1990).
Split attention effects can also be reduced by utilizing modality effects. Modality effects
can be obtained when we increase the effective size of working memory by utilizing different
types of working memory processors (Jeung et al., 1997), derived from a popular assumption in
memory research that there are multiple working memory stores, also called multiple memory
channels or processors. A typical hypothesis of dual processing modalities is that students who
29
receive verbal information in narration along with animation or diagrams simultaneously or
sequentially outperform those who receive the verbal information in an on-screen text format,
when other things being equal (Moreno & Mayer, 1999, Experiment 2; Mousavi et al., 1995). By
presenting an explanation in an auditory format learners can utilize the verbal processing channel
in working memory as well as the visual channel, which eventually increase limited working
memory capacity.
Jeung et al. (1997) also agreed that mixed mode of presentations (e.g., visual diagrams
and auditory text) would have more working memory available for learning compared to a single
mode of presentations of equivalent contents (e.g., visual diagrams and written text). Yet they
took a further step that clearly has very significant implications in designing instructional
multimedia. They hypothesized that a multimodal presentation would be useful only when
learners do not have much difficulty with relating audio and visual information. Furthermore,
they also suggested that when learners have to invest a high level of cognitive resource to search
for connections between two sources of material due to the complexity of the diagram, a visual
indicator such as flashing or color would reduce learners’ search effort so that cognitive resource
could be used for learning the material.
Leaving out redundancy effects
A common assumption in multimedia learning is that different instances of the same
information, or simply stated, repeated information in different modalities would enhance
learning by allowing learners to choose the presentation mode that best fits there learning
preferences. However, it has been repeatedly shown that if two different segments of information
can be understood in isolation, one of them is redundant, and removing this redundant
information has positive effect on learning and performance (Bobis, Sweller, & Cooper, 1993;
30
Chandler & Sweller, 1991; Harp & Mayer, 1997, 1998; Mayer et al., 2001; Sweller & Chandler,
1994, Experiment 2). Moreover, Chandler and Sweller (1991) revealed, not only redundant
information is unnecessary for learning, but also it in fact interferes with the learning of core
information by leading learners to pay attention to redundant material, and by imposing
extraneous cognitive load on working memory.
Another major source for redundancy effect is adding interesting but conceptually
irrelevant material, commonly defined as seductive details, to a lesson in an attempt to induce
learner interest. Seductive details are redundant in that they are in many cases added to entertain
learners not because they are essential to the acquisition of core elements of a lesson. The
underlying premise of including entertaining elements in a multimedia presentation is that
increased learner interest will lead learners to pay more attention, to persist longer to learn, and
to exert more effort to process instructional material. Moreno and Mayer (2000a), however,
found that adding entertaining, but irrelevant to learning, music and sounds interfered with
learners’ recall and understanding of the lesson. Any additional elements that are not crucial for
understanding of core material decrease working memory capacity, and as a result, learners are
left with few cognitive resources to use for processing core elements. The result is poorer
performance.
Research also points out that including seductive details that lack conceptual relevance
to main ideas of the lesson hinders learning process since they could take learner attention away
from selecting and processing key elements of the lesson; causing split-attention effect (Mayer et
al., 2001). In addition, entreating but irrelevant information may prime inappropriate knowledge
about the topic and reduce the amount of cognitive resources that learners can use for processing
essential information. Entertaining elements are most seductive when they are novel, active,
31
concrete, and personally interesting (Garner, Brown, Sanders, & Menke, 1992) such as animated
graphics and characters embedded in computer-based lessons.
Integrating Leaner Prior Knowledge and Expertise
Although the theories and research findings discussed so far are very much
comprehensive and convincing, there is still one missing component, that is, learners. As many
educators have argued, learners should be and in fact, are active participants of their own
learning (Dalgarno, 2001; Wertsch & Bivens, 1992). They faithfully play their part in learning by
bringing varying degrees of prior domain knowledge and expertise although often ignored by
instructional designers. In effect, it is reported that differences in the levels of learner expertise
make very significant differences in cognitive processing and performance (Ericsson & Charness,
1994). According to Kalyuga et al., (1998), there are two important concepts to understand
learner expertise and prior knowledge, ‘Schema’ and ‘Automation’. They define schema as “a
cognitive construct that permits people to treat multiple sub elements of information as a single
element, categorized according to the manner in which it will be used” (p.1). Thanks to s schema,
which is stored in our long-term memory, we can identify various kinds of dogs as a dog, and as
a consequence, avoid overburdening working memory. One of the important characteristics of
schemata is that they are transferable (van Gerven et al., 2000). That is, we can transfer an
existing schema to new problems as far as they are related to the earlier encountered problems.
Another feature of schemata is automation. Automation decreases working memory load
(Kotovosky & Simon, 1985; Shiffrin & Schneider, 1977), and requires little cognitive resources
since automated knowledge can be processed unconsciously. Schemata are saved in long-term
memory with varying degrees of automaticity, and depending on the degree of automaticity,
schemas require a different amount of working memory resources. Therefore, if a learner faces
32
with a problem for which he or she has a fully automated schema, then the learner does not have
to spare cognitive resource for organizing and categorizing the problem. Instead, more conscious
effort can be used to search for a solution and performing a task, and thus, a better and faster
learning process can be achieved. Keeping the effects of learner expertise on learning in mind,
we may have to ask the following questions in designing multimedia learning environment: Do
seemingly redundant learning materials have the same effect on every learner regardless of their
level of expertise? Or in reality are they redundant to only some people?
Kalyuga, Chandler, and Sweller (2000) reported that while inexperienced learners
benefit from dual-mode presentation (i.e., a diagram with auditory narration), more experienced
learners do not need additional source of information since they have already acquired sufficient
knowledge to fully understand one source of information. Furthermore, it was found that
experienced learners actually were able to skip the extra information, the auditory narration,
when given the options to do so, and thereby there was no redundancy effects found. Yet, under
the condition in which experienced learners were forced to attend to both auditory and visual
information, their performance became worse because they had to process redundant information,
which probably imposed extra cognitive loads on learners’ limited working memory and required
unnecessary mental effort investment. The authors concluded that multimedia instructions have
beneficial effects on learning only under well-defined circumstances, and in particular, learners
should be given options to adapt themselves to a learning environment in order to take into
consideration differing levels of learner expertise.
Summary
In this section, the construct of cognitive efficiency and its importance in multimedia
learning was discussed. In addition, several instructional strategies were reviewed that can be
33
used to prevent working memory from being cognitively overloaded and to require less mental
effort from learners in multimedia learning environment. Sweller and Chandler (1994) suggested
that by avoiding the limitations of working memory it is possible to increase the efficiency of
learning process. In learning situations, therefore, the total amount of cognitive load imposed by
instructional materials should not exceed the limited capacity of working memory to speed up
learning process and to achieve successful performance.
Moreover, it has been shown that learner prior knowledge is also an important factor to
be considered when designing multimedia-based instruction since different degrees of learner
prior knowledge have different effects on learning processes and outcomes as well. Hence, it is
necessary to examine the role of learning prior knowledge in cognitive efficiency of multimedia
learning, and the present study hypothesizes that there will be a significant positive correlation
between the level of learner prior knowledge of L2 grammar and the level of cognitive efficiency.
In other words, the more prior knowledge learners have of a target form, the less mental effort
and time they will spend to process learning material.
Another important point has been made in this section about adding interesting but
conceptually irrelevant material to a lesson in an attempt to induce learner interest. A body of
research has shown that any additional elements that are not crucial for understanding of core
elements decrease working memory capacity, and as a result, learners are left with few cognitive
resources to use for processing core elements. Research also points out that any additional
elements which are entertaining but unrelated to learning could take learner attention away from
selecting and processing key elements of the lesson; causing split-attention effect. This point is,
in particular, closely related to the next section on animated pedagogical agents because the
major reason behind using APA’s is to entertain students and direct attention to instructional
34
elements. Yet, given that entertaining elements could also split learner attention and simple
flashing arrows can assist learners to focus and integrate related elements (Jeung et al., 1997),
the effect of APA’s on learning processes and outcomes should be carefully examined and
compared with other alternative ways. The present study hypothesizes that there will be no
significant difference between APA’s and flashing arrow in learner performance when learning
L2 grammar, but the flashing arrow will require less time and mental effort resulting in better
cognitive efficiency.
Animated Pedagogical Agents
An animated pedagogical agent, by definition, is a lifelike character that resides in an
interactive computer-based learning environment. It is a product of recent technological
advances in user interface and autonomous software agents, and has been claimed to have great
potential in facilitating human learning by offering customized instruction, and context-specific
feedback and advice to learners (Johnson et al., 2000; Moreno, Mayer, & Lester, 2000; Samson,
Karaginaanidis, & Kinshuk, 2002). It is also one specific type of intelligent interface agents
whose main purpose is to provide learners with pedagogical assistance. In other words, an
animated pedagogical agent is a specific form of media used to present instruction, not an
instructional method itself, and thereby what an animated pedagogical agent does can be
delivered through other media with different degrees of efficiency and effectiveness.
The field of animated pedagogical agent has originated from two research areas:
‘Animated Interface Agent’ and ‘Intelligent Tutoring System’ (Johnson et al., 2000). The
research on animated interface agent provides a new way of human-computer interaction by
applying the features of face-to-face human communication to human-computer interaction. This
35
field has had great impact on the technological development and design of animated pedagogical
agents, especially, its emphasis on lifelike and believable agent behaviors, such as gesture, facial
expression, and gaze. On the other hand, the intelligent tutoring system focuses on developing
software that can adapt to individual learners and provide personalized feedback through the use
of artificial intelligence. From this, the field of animated pedagogical agent has taken on its
capability to provide personalized tutoring experience by keeping track of student progress over
time and accordingly generating pedagogical actions to meet the needs of individual students. By
incorporating these two areas of research, animated pedagogical agents are believed to enhance
computer-based learning, especially affective and motivational aspects of learning experiences
(Atkinson, 2002).
Animated pedagogical agents have a range of functions depending on the environment
which they inhabit: (a) it can provide a learner with opportunistic instructions through which
they can respond and adapt dynamically to the surrounding environment including the learner
(Moreno et al., 2002): (b) it can interactively demonstrate how to perform tasks (Samson et al.,
2002): (c) it can focus learners’ attention on certain elements or aspects of instructional systems
with common and natural methods, such as gestures, locomotion, or gaze (Atkinson, 2002): and
(d) It can provide nonverbal as well as verbal feedbacks on learners’ actions. In particular, the
capability of using nonverbal communicative behaviors allows the tutoring system to provide
less obtrusive feedback (i.e., head-nodding for approval, head-shaking for disapproval, jumping
up and down for congratulating students’ success, look of puzzlement for misunderstanding)
(Johnson et al., 2000).
Benefits of Animated Pedagogical Agents
A number of instructional benefits have been claimed and studied on APA’s although no
36
one agent has provided all of the benefits to date (Johnson et al., 2000). A review of literature
shows that animated pedagogical agents in general have three possible effects on learnercomputer interaction: (a) it can have a positive impact on learners’ motivation and perceived
experience of interaction with the system: (b) it can attract learners’ attention on the system or
tasks through the use of motion, gesture, and facial expression, and: (c) it can also improve
learning outcomes (i.e., problem solving, understanding of knowledge) by providing learners
with contextualized advice. In particular, advocates of animated pedagogical agents maintain that
agents render learning environment entertaining, which in turn motivate learners to interact more
and to stay longer in the system. Due to the increased motivation and interaction, it is assumed
that learner performance would improve. Among these three effects, however, the first two
effects have close bearing with the present study, and thus, in the following section, they will be
discussed in relation with learning process and outcomes based on empirical research findings.
Motivating Learners
One of the biggest benefits of using pedagogical agents put forward is that agents can
entertain and motivate students better than other media and consequently, lead students to exert
more efforts to make sense of instructional material. This claim is based on the interest theories
of motivation (Harp & Mayer, 1998), which suggests that learners invest more effort when they
are interested in the presented learning material. A similar effect, called “Persona Effect” has also
been the focus of research in the field. Precisely, the persona effect refers to learners’ positive
perception of their learning experience caused by the presence of the agent (Moreno et al., 2001).
The persona effect is derived from the hypothesis that people interpret their interaction with a
computer or a computer-mediated figure as a social interaction (Reeves & Nass, 1996) and they
form personal and emotional connection with a computerized character. This personal and
37
positive feeling is believed to foster interest in learning tasks and lead learners to work harder
(Lester, Converse, Kahler, Barlow, Stone, & Bhogal, 1997). Although these two effects have
originated from different theoretical backgrounds, they will be discussed together in the
following since they are concerned with similar phenomena – learners’ subjective experiences of
learning
Numerous claims have been made on the positive influence on learning brought by
increased motivation and positive perception. Yet, it should be pointed out that increased
motivation or positive perception does not necessarily have a cause and effect relationship with
students’ actual learning outcomes. In other words, although students enjoy their interaction with
an agent, it does not mean that they will learn better (Lester, Voerman, Towns, & Callaway,
1997). In fact, many studies which employed animated pedagogical agents to deliver instruction
found the persona effect, but were not able to find any significant cognitive learning benefits of
the agents. In the studies where better learning outcomes were found for the students who
interacted with agents, the benefits were more likely caused by either different levels of advice or
different instructional methods rather than higher interest or motivation triggered by the presence
of agents (Clark & Choi, 2003; Dehn & van Mulken, 2000).
The lack of increased learning outcomes in spite of high levels of interest elicited by
agents could be explained by the theories of individual and situational interests. Individual
interest refers to a relatively enduring predisposition towards certain objects or events, which
develops slowly overtime (Ainley, Hidi, & Berndorff, 2002). Situational interest, on the other
hand, is psychological state triggered by immediate environmental stimuli, and as a result, may
not have lasting influence on personal interest or learning (Hidi & Anderson, 1992). Given that
an animated pedagogical agent is an environmental stimuli embedded in a computerized learning
38
environment, not a learner-developed characteristic, it can be inferred that learner interest
generated by an agent might not last long enough to have significant impact on learning outcome.
It also could be explained by the novelty effects (Clark, 1983).
One of the studies that examined the motivational effect of an agent on learning is
Moreno et al.’s (2001). They investigated whether learners make more effort to understand
material and accordingly achieve deeper learning in a ‘social agency environment’ (Experiment 1
and 2). Participants in the experiment group interacted with a pedagogical agent, ‘Herman’, an
alien bug residing in a discovery- and design-based learning environment called ‘Design-A-Plant.
The experiment group participants were asked to design a plant based on environmental
conditions (e.g., the amount of sunlight and rainfall), and received verbal feedback from the
agent on their choices. In contrast, participants in the control group were given onscreen text
which explained about making right choices under specific environmental conditions using stepby-step worked examples instead of the agent. But they were not allowed to design plants by
themselves unlike their counterpart in the experiment group.
In the immediate posttests, the researchers found significant learning differences in
problem solving questions between the agent and non-agent conditions while found no difference
in the retention questions. Their interpretation of the results was that students interacting with the
agent felt personal connection with the agent and interested in the task, which in turn resulted in
more effort and better performance. However, it should be pointed out that the two groups
received different instructional treatments in addition to presence or absence of the agent. Only
the agent group had opportunity to design plants and received contingent feedbacks, and thus the
better performance of the agent group cannot be exclusively attributed to the presence of the
agent. Moreover, although the agent group indicated higher interest in the material than the non-
39
agent group, the groups did not show any significant difference in their subjective ratings of how
difficult or understandable the material was, which can be related to the amount of mental effort
they put during the learning (Salomon, 1984). Moreover, it should be mentioned that the amount
of mental effort invested by the participants during learning was not measured, and their claims
were solely based on the results of simple t-test on the levels of self-reported interest of two
groups.
In a study using five clones of Herman the alien bug, Lester et al. (1997) also examined
affective impact of animated pedagogical agents. The five clones were identical in their
appearance, voice qualities, and nonverbal communicative behaviors, but were different in the
levels of instructional advice and feedback they provided (task-specific vs. principle-based), and
in modalities they employed to deliver instruction (verbal-only or verbal and animated). The
students who interacted with the agent having full functionalities and modalities produced
highest performance and gave the highest ratings to subjective assessment questions asking about
helpfulness, believability and utility of the agent’s advice. Based on the simple analyses (i.e.,
means, standard deviations), the researchers interpreted that enhanced learning was likely to
result from students’ positive perception of learning experiences and increased motivation,
despite the obvious differences in instructional treatments.
Unlike the previous two studies that failed to control the effect of methods and
feedbacks, Andre, List, and Muller (1999) conducted a well controlled agent study. To find
empirical support for the affective and cognitive effects of the PPP Persona on man-machine
communication, they exposed participants to a technical description (the operation of pulley
systems) and an informational presentation (names, pictures, and office locations of fictitious
employees) on the Web. Both experiment and control versions provided the same treatments
40
except that the control groups did not have the PPP Persona; instead, a voice was employed to
convey the same explanations as the Persona, and its pointing gestures were replaced with an
arrow. The Following the presentations, the Persona’s affective effect was measure through a
questionnaire whereas the cognitive impact was measure by comprehension and recall questions.
The results showed significant differences in only the affective measures. Participants
interacting with the Persona for the technical descriptions found the presentation less difficult
and more entertaining. These positive effects, yet, were not found for the informational
presentation about the fictitious employees; rather subjects reported that the Persona was less
appropriate for the domain and less helpful as an attention direction aid. As for the learning effect,
no significant difference was found between the Person version and the non-Persona version both
in the technical domain and the non-technical domain. The results of this study is consistent with
Dehn and van Mulken’s claim (2000) that the persona effect of an agent is domain-specific and
can improve human-computer interaction if the agent displays the functional behaviors matching
the system’s purposes. The findings also suggest that Persona behaviors can be easily replaced
by simpler means of communication, not necessarily requiring an embodied character.
For example, Erickson (1997) argued that the adaptive functionality of an instructional
system is often enough for learners to perform a task and achieve the same outcomes without the
guidance of an agent. He also suggested that when including an agent, instructional designers
should think about what benefits and costs the agent would bring, and far more research be
conducted on how people experience agents. Furthermore, Nass and Steuer (1993) found that
simply using a human voice was sufficient to induce learners to use social rules when interacting
with a computer. Moreno and colleagues (2001) also noted that learners may form a social
relationship with a computer itself without the help of an agent and thus, the image of an agent
41
might not be necessary to invoke a social agency metaphor in a computer-based learning
environment.
Given the studies discussed above, it is premature to conclude that the enchanting
presence of an agent causes instructional benefit unless an agent condition is compared with a
non-agent condition that provides the exact same learning conditions including the types of
instructional methods, and the levels of feedback and advice. In fact, Moreno et al. (2001) found
that students who interactively participated in designing plants and received contingent feedback
on their choices still learned better than those who just passively listened to the verbal
explanation, even after the image of the agent was deleted from the screen and the voice of agent
was replaced by human voice (Experiment 3). What is more interesting is that students in two
groups did not show any significant difference in their perception of learning experiences, which
was speculated to be the main cause of learning outcomes. That is, even without the benefit of
the presence of an agent, students were still able to learn the material better if an instructional
method was right for the task. Moreno et al. also found that the presence of the agent’s visual
image did not have any impact on affective or cognitive aspect of learning, while the modalities
of the agent made significant differences in learning outcomes (Experiment 4 and 5). These
results may imply that what makes a difference in student learning is not the agent itself or
increased motivation or positive perception caused by the agent, but rather the level of
interactivity and contingent feedback as well as modalities, which will be empirically tested in
the present study.
Focusing Learner Attention
According to cognitive load theory, the presence of an animated pedagogical agent can
be detrimental to learning by dividing a learner’s limited cognitive resources into different visual
42
segments. More specifically, cognitive load theory predicts that when animation of an agent is
presented simultaneously with other visual information such as graphics or texts, learners need to
split their attention between these two sources, and as a consequence, the presence of the agent
becomes harmful to learning rather than beneficial. The split attention effect could be even worse
when the agent’s dialogues are presented as onscreen text since both the animation and text
require learners’ limited visual resources (Moreno et al., 2001). However, even replacing
onscreen text with spoken text may not be enough to overcome the split attention effect if
learners have to mentally connect the visual information the agent is presenting and aural
information delivered through the spoken text (Jeung et al., 1997).
In order to prevent the split attention effect caused by the presence of an agent, it is
suggested that animated pedagogical agents use non-verbal behaviors (i.e., pointing gesture,
jumping to the target object) in order to draw learners’ attention to relevant learning material
(Atkinson, 2002). By focusing a learner’s attention to pertinent segments of the lesson, the agent
can connect auditory and visual information for learners, thus freeing learners’ cognitive
resources to be used for solving problems or understanding material. Yet, there is a possibility
that agents’ attention-focusing behaviors can act as seductive details in that they are not
conceptually related to primary learning objectives but still catching learners’ attention.
Additionally, despite that agents’ behaviors are supposed to help learners pay attention to the
lesson, learners still have to spare their limited cognitive resources to understand their behaviors
and facial expressions. Nevertheless, studies have shown neutral effects of presence of agents,
neither providing negative nor positive impacts on drawing learner attention (Atkinson, 2002;
Moreno et al., 2000).
Using worked examples to instruct proportion-word problem solving, Atkinson (2002)
43
examined the effect of presence of an animated pedagogical agent and delivery modes (e.g.,
voice or text) on learning. In the study, the animated agent was programmed to deliver
instructional explanations which contain elaborated information regarding solution steps for
word problems, either aurally through human voice or textually through a cartoon-like word
balloon appearing above the agent’s head. Yet, the contents of explanations were identical for
both conditions. The agent was also programmed to link the explanations with the respective
visual information on the screen by utilizing nonverbal communicative behaviors (i.e., pointing
gesture, glancing toward a solution step).
The analyses of learning-process and learning-outcome measures revealed that
participants in the voice plus agent group outperformed those of the text only group in perception
of difficulty of practice problems, near- and far-transfer problems and affective measures of the
learning environment, while their performance was superior to that of the voice only group only
in far-transfer problems. Atkinson interpreted this result relatively positively insisting that
presence of an agent did not distract learners’ attention. However, the fact that the voice only
group performed as well as the voice plus agent group in most assessments suggests that we
might not need the image of an agent to achieve similar learning criterion. Moreover, given that
the voice only group did not have the benefit of visual indicators connecting aural information
with appropriate visual information, it is not very clear what proportion of the learning benefits
can be attributed to the agents’ attention-directing behaviors. And again what should we choose if
other visual indicators rather than the agent’s nonverbal behavior, such as electronic flashing,
could have the same level of effect on focusing learner attention, but are a lot easier and more
economical to create?
Craig, Gholson and Driscoll (2002) provide a possible answer for the question proposed
44
above. They designed a 3 (agent properties: agent only, agent with gestures, no agent) x 3
(picture features: static picture, sudden onset, animation) study to investigate issues concerning
the ways to capture learners’ attention. For the agent property, they investigated: 1) whether the
use of an agent leads to split attention; and 2) whether integration of the agent’s gestures with a
picture or animation helps to direct learners’ attention. For the picture features, they tested if
using parts of a picture or animation captures learners’ attention. In the experiments, participants
learned the process of lightning formation presented through an agent and multimedia material
(e.g., pictures, narration, or animation). The narrative information was synchronized prior to or
simultaneously with the agent’s pointing gestures toward, or with a sudden onset (e.g., color
singleton or electronic flashing) or animation of relevant parts of a picture.
Again, they did not find the Persona effect, that is, participants did not find learning with
an agent particularly more interesting than that without an agent. Moreover, the agent made no
difference in learners’ performance both in cognitive load assessment and performance tests
(retention, matching, and transfer). On the contrary, Craig and his colleagues found significant
effect of both a sudden onset and animation of parts of the pictures for focusing learners’
attention. This result can be taken as evidence that a well integrated pedagogical agent is not
harmful to learning, but not beneficial either. It can be also inferred that there is a far simpler and
easier way (coloring and flashing) to have the same effect rather than using a technically
complicated agent. To clarify the confusion about the relative effect of a pedagogical agent for
directing learner attention, the present study compares an agent with an electronic flashing arrow.
Summary
The review reveals that while there is strong enthusiasm persisting in the field regarding
the motivational and instructional effects of animated pedagogical agents in multimedia learning
45
environment, very few empirical studies have been conducted so far and the results of these
studies are rather neutral or sometimes negative. The literature does not support the assumption
that the motivational effects of animated pedagogical agents lead to better performance in
learning including problem solving and knowledge retention (Dehn & van Mulken, 2000). The
optimistic perspective of the field is largely based on the hope that rapidly advancing technology
will be able to fix the existing technical limitations that prevent an agent from fully imitating a
human tutor (e.g., recognizing individual learners’ learning styles and specific characteristics and
thereby providing more contextualized and individualized feedback).
As a matter of fact, a majority of the animated pedagogical agent studies published to
date have focused on technological capacity of agents without paying much attention to theoryand research-based principles (Bradshaw, 1997). Yet, it should be pointed out that there are some
aspects of these studies that cannot be fixed by advanced technology, including methodological
shortcomings (e.g., theoretical foundation, study design, variable manipulation, and
measurement of effects). For instance, only few studies did compare the agent-embedded
environment with the non-agent environment on an equal basis. In many cases, there were other
significant factors in addition to the agent which could have contributed to differences in
learners’ attitudes and learning. The instruments employed to measure the effects of agents have
posed another major problem. Several studies failed to provide the validity and reliability of the
instruments they used to measure effects, which consequently undermined researchers’
interpretation with regard to agent effects. At the point, thus, we need a more theory-based
approach to investigate agent-embedded learning environments, and more structured research
methodologies to investigate the effects of these environments.
What is clear about the field of animated pedagogical agent is that due to its infancy of
46
development, only a small number of studies have been conducted, and even worse, not many
studies conducted so far have employed systematic and thorough research methods (e.g., a small
sample size, confounding variables, unreliable measuring instruments). In addition, the fact that
only a small number of knowledge domains have been the focus of agent studies does not help to
understand what kind of domain could benefit most from animated pedagogical agents with what
kind of functionalities. For example, most pedagogical agent studies have employed discoverybased learning environment and scientific materials for instruction even though there is evidence
that entertaining effect of an agent is domain specific (van Mulken, Andre, & Muller, 1998).
Consequently, it is not clear that what effects would be gained in different environments with
different subject matter. The present study is expected to contribute to the field in this aspect
since it examines the effect of pedagogical agents for learning L2 grammar, a rarely explored
subject matter.
Research Hypotheses
The review of literature in three research areas – second language learning, multimedia
learning, and animated pedagogical agents – is the foundation for this study. The primary
objective of this study is to solve the confusion between instructional methods and animated
pedagogical agents, and to empirically examine cognitive efficiency of different media in
learning second language (L2) grammar in a computer-assisted learning environment. Learners’
focal attention to target grammatical elements is required for learning of L2 grammar (Rosa &
O’Neill, 1999) because only the attended form can be perceived and processed by learners.
Several instructional methods have been developed to direct learner attention to target linguistic
features. The present study will use two instructional methods to focus learners’ attention on a
47
target L2 grammatical form, English relative clauses: explicit metalinguistic explanation about
the target form and its use, and visual enhancement of the form embedded in input text.
The first instructional method, the metalinugistic rule explanation, will be presented in
an aural mode rather than in a text mode to avoid the split attention effect between the
explanation and the main text. For the visual enhancement treatment, the present study will make
use of two computerized multimedia techniques: an animated pedagogical agent and a flashing
arrow. These two multimedia delivery methods will be compared on the extent to which they
improve cognitive efficiency in acquiring an L2 grammar, while the two instructional methods,
the explicit explanation and the visual enhancement, will be compare in terms of their effects on
learning outcomes. The research hypotheses of this study are explained in detail below.
Hypothesis 1 - There will be a significant difference in learner performance between two groups
that receive different instructional methods.
1) Participants who receive explicit explanation on the target form will significantly
perform better in comprehension and production tests on the target L2 grammar than
those who receive the visual enhancement treatment only.
2) Participants who receive explicit explanation on the target form and also interact with
one of visual enhancement treatments will significantly perform better in comprehension
and production tests on the target L2 grammar than those who only receive either explicit
explanation or visual enhancement treatment.
The general consensus in the field of second language acquisition (SLA) is that attention
to the target grammatical element is necessary for learning to take place. The literature identifies
two major instructional methods to draw learners’ focal attention to the target L2 grammar:
visual enhancement and explicit explanation. The former employs more unobtrusive techniques
48
such as underling or highlighting the target grammar, and the implicit methods are believed to
help learners focus not only on the form, but also on the meaning of the text. On the contrary, the
latter focuses more on providing metalinguistic rule explanations about the target form and its
use. Recent studies show that the explicit attention-directing method works better than the
implicit method, and accordingly, learners who are explicitly directed to the target grammar
perform better than those who are implicitly exposed to the target form. It is because the explicit
rule presentation better primes learners for a specific grammatical form (Alanen, 1995). Research
further reveals that students who receive both implicit and explicit attention-directing treatments
perform better than those who receive either treatment only because L2 learners need not only
understand the form but also they have to integrate the form with meaning (Ellis, 1994).
Hypothesis 2 - There will be no significant differences in learner performance measured by
comprehension and production tests on the target L2 grammar between participants who interact
with an animated pedagogical agent and those who interact with a flashing arrow.
What causes learning is an instructional method, not a delivering medium (Clark, 1983,
1994a, b, 2001). Therefore, there will be no significant difference in learner performance on the
target L2 grammar among participants who receive the same instructional method that are
delivered through different media. The instructional methods employed in this study to focus
learner attention are explicit explanation and visual enhancement of the form. When the
instructional methods are held constant, the final product of learning, the comprehension and
production of the target grammar, will be the same regardless of what medium is used to deliver
the instruction. Therefore, the two multimedia techniques used to deliver the visual enhancement
treatment, the animated pedagogical agent and the flashing arrow, will not make any significant
differences in the final learning outcomes.
49
Hypothesis 3 – The more prior knowledge of the target L2 grammar learners have, the less effect
of instructional methods will be obtained on learner performance.
1) Participants with less prior knowledge on the target form will benefit from both explicit
explanation and visual enhancement more than those with more prior knowledge.
2) The benefit of explicit explanation will decrease as participants’ levels of expertise in the
target L2 grammar increase.
3) The benefit of an animated pedagogical agent and a flashing arrow will decrease as
participants’ levels of expertise in the target L2 grammar increase.
The two instructional methods employed in this study will significantly enhance learner
performance when learners have a low level of prior knowledge on the target L2 grammar.
Research shows that explicit rule explanation and visual enhancement have positive effect on L2
grammar acquisition by sensitizing learners to the target linguistic features embedded in the text.
Given that explicit explanation provides information not only about the structure of the target
grammar but also about the use of the target grammar, it will help learner build a schema for the
target grammar. The visual enhancement treatment will also have significant impact on learner
performance since it shows the relationship among relevant elements of the target grammar as
well as it makes the target grammar perceptually salient. However, once learners acquired the or
schema about the target grammar, the benefit of the instructional methods will decrease or
disappear. The acquired schema allows learners to identify the target grammar in the text without
the information provided by the instructional methods (Kalyuga et al., 1999). Furthermore, once
learners understood the relationship among relevant elements of the target grammar, the
guidance provided by the animated agent or the flashing arrow will not be needed.
Hypothesis 4 – There will be no causal relationship between learners’ levels of interest in
50
instructional program they interacted with and the level of learner performance measured by the
comprehension and production tests on the target grammar.
The underlying premise of including entertaining elements such as an animated
pedagogical agent in an instructional presentation is that the increased learner interest triggered
by entertainingness of the system will lead learners to pay more attention, to persist longer in the
system, and to exert more effort to process the information. Although numerous claims have
been made about the positive influence of animated pedagogical agents on motivation and
interest, it should be noted that increased motivation or interest does not necessarily have a cause
and effect relationship with students’ actual learning outcomes. In other words, although students
enjoy their interaction with the agent, it does not guarantee that they will learn better than those
who do not interact with an agent. For instance, a number of studies which claimed any learning
effects of animated pedagogical agents in fact provided different instructional treatments to agent
groups and non-agent groups. And as a result, it is hard to attribute the benefits exclusively to
animated pedagogical agents (Clark & Choi, 2003; Dehn & van Mulken, 2000).
Hypothesis 5 - Participants’ levels of attention directed to the target L2 grammar during the
processing of learning material will have significant effect on the level of learner performance
measured by the comprehension and production tests on the target grammar.
It is generally agreed that attention to a certain linguistic feature, the target of instruction,
is required for learning to take place. Although there has been a controversy on how much and
what type of attention to the target form is necessary for learning (Izumi, 2002), it is now widely
recognized that conscious, focal attention is necessary for learning, and a higher level of
subjective awareness is correlated with better learning (Rosa & O'Neill, 1999). Following
Schmidt’s Noticing Hypothesis, attention is operationally defined as ‘the availability for
51
verbalizing what one has experienced during or right after exposure to the input’ in the present
study.
Hypothesis 6 – The flashing arrow will require less time and mental effort from participants than
the animated pedagogical agent in achieving the same level of learning performance, when both
media are used to deliver the visual enhancement treatment.
Different media employed to deliver the same instructional method will have significant,
different effect on levels of cognitive efficiency, defined as ‘the relative amount of time and
mental effort required by different media to reach a learning criterion’ in the present study. The
underlying premise of cognitive efficiency is that a specific medium used to present instruction
may not produce different cognitive outcomes compared to another medium, but it still can have
direct impact on cognitive processes. Participants receiving the flashing arrow will need to invest
less mental effort than those interacting with an animated pedagogical agent in order to reach the
same level of learning performance. Moreover, participants interacting with the flashing arrow
will achieve the same level of learning outcome significantly faster than those interacting with an
animated pedagogical agent.
An animated pedagogical agent is seductive details in that it is active, concrete, and
interesting but conceptually irrelevant to learning of the target form, and consequently, it takes
away learners’ visual attention from the instruction. Even though the animated pedagogical agent
in this study is used to indicate the relationship among relevant elements, its presence and eyecatching behaviors (i.e., gestures, lip movements, locomotion) will demand learners’ conscious
attention and impose unnecessary, extra cognitive load on working memory. On the contrary, the
flashing arrow will achieve the same effect of indicating the relationship without overloading
learners’ limited working memory since it is integrated in input text and does not require
52
participants to draw their attention away from the text.
Hypothesis 7 – There will be a significant positive correlation between the level of learner prior
knowledge of the target grammar and the level of cognitive efficiency.
According to Cobb (1997), learner prior knowledge or schema is one of the major
contributing factors to cognitive efficiency. The more prior knowledge learners have of the target
form, the less they will invest mental effort and time to process learning material. Schema helps
learners distinguish relevant target elements from irrelevant ones and then integrate the new
information into the existing knowledge of the target form (Kalyuga et al.,1998). Therefore,
learners who have already developed appropriate schema of the target form will not need extra
help from attention-directing techniques for processing the instruction, and will be able to avoid
spending limited cognitive resources in integrating information. Instead, more conscious effort
can be used to search for a solution. As a consequence, they will be able to learn faster and/or
easier than their counterparts who have not developed appropriate schema yet. However, the
strong presence of an animated pedagogical agent could force even experienced learners to
attend to unnecessary information provided the agent. This will probably impose extra cognitive
load on learners’ limited working memory and require unnecessary mental effort investment. As
a result, their performance will be damaged and the level of cognitive efficiency will decrease.
53
CHAPTER II: Methodology
Target English Grammar for Instruction
English Relative Clauses will be used as a target form in this study. It is reported that
English relative clauses are hard to comprehend and produce for both L1 and L2 speakers, and
instruction can facilitate acquisition of the form (Izumi, 2000).
Participants
120 Participants will be recruited from students enrolled in ESL (English as a Second
Language) programs at local universities and community colleges. The selection of participants
will be determined based on a test of English relative clauses, which will be also used as a pretest
and the indication of students’ prior domain knowledge. Following Doughty (1991) and Izumi
(2002), among participating students, those who only have a basic level of knowledge of English
relative clauses (although it is expected that there will a wide range of proficiency levels among
these students) will be selected to participate in experiments. Students who have advanced
knowledge of the target form or those who have no knowledge at all will be eliminated from the
present study.
Research Design
The design of the study will be a true experimental with pre- and post test involving five
treatment groups and one control group, as can be seen in Table 1. The treatment groups will be
different with regard to provision of explicit explanations (+EE) on the target form and the
provision of visual enhancement technique (+VE) to focus learners’ attention on the target form.
54
The participants will be randomly assigned in equal proportion to one of the six conditions:
a) Explanation plus agent (+EE +PA) group will receive explicit explanation by the agent;
b) No explanation plus agent (-EE +PA) group will be provided with the visual
enhancement treatment through the agent;
c) Explanation plus flashing (+EE +FA) group will receive explicit explanation through
human voice with the flashing arrow used to focus attention;
d) No explanation plus flashing (-EE +FA) group will not receive any explanation, only the
flashing arrow will be used;
e) Explanation plus no attention drawing method group (+EE –VE) will only get explicit
explanation;
f) No explanation plus no attention drawing method group (-EE –VE), or the control group
will not get any instructional support.
Table 1
Experimental and Control Conditions
Treatment A (Rule Presentation)
Treatment B
(Visual
Enhancement)
APA
Flashing
Arrow
No Visual
Enhancement
Explicit Explanation
No Explicit Explanation
+EE +PA
-EE +PA
+EE +FA
-EE +FA
+EE -VE
-EE –VE
Measures
There are the total of five dependent variables in the present study which include mental
effort measures, time measures, interest measures, attention measures, and performance
55
measures. These measures will be used to investigate the research hypotheses discussed above.
In order to measure the amount of mental effort that learners invest in processing the instruction,
the secondary task method will be embedded in the learning environment which will be designed
specifically for this study. Learners’ interaction with the system will be logged online including
their entry and exit time to measure how much time is required for individual learners to acquire
the target form. After interacting with the system, participants will be asked to fill out two
questionnaires; the first one consists of questions measuring their interest in the system while the
second one inquires whether students were able to focus on the target form and its relationship
with other elements. Finally, participants will be required to take the post test to find out the
amount of their learning outcomes brought by the treatments used in the study. The post test will
be composed of questions determining learners’ levels of capability to comprehend and produce
the target form.
Data Analysis
Descriptive statistics will be used throughout the study to estimate all measures.
Additionally, Multiple Regression and MANOVA will be used to investigate the relationship
among mediating (i.e., learner prior knowledge), process (i.e., time required to acquire the form,
the amount of attention paid to the form), and outcome measures (i.e., scores for comprehension
and production tests). Details are given in the following about how each hypothesis will be tested
and analyzed.
Hypothesis 1 - There will be a significant difference in learner performance between two groups
that receive different instructional methods.
1) Participants who receive explicit explanation on the target form will significantly
56
perform better in comprehension and production tests on the target L2 grammar than
those who receive the visual enhancement treatment only.
2) Participants who receive combination of explicit explanation on the target form and
visual enhancement treatments will significantly perform better in comprehension and
production tests on the target L2 grammar than those who only receive either explicit
explanation or visual enhancement treatment.
Descriptive statistics (e.g., means, standard deviations) will be adopted to assess each
group’s posttest performance measures. In addition, 2 (Explicit explanation, no explanation) x 2
(Agent, Flashing) ANOVA will be performed to test the significance of differences among
groups.
Hypothesis 2 - There will be no significant difference in learner performance measured by
comprehension and production tests on the target L2 grammar between participants who interact
with an animated pedagogical agent and those who interact with the flashing arrow.
Descriptive statistics (e.g., means, standard deviations) will be used to estimate the
outcome measures on post tests, and T-test will be used to test the significant difference between
two groups’ scores on comprehension and production tests.
Hypothesis 3 – The more prior knowledge of the target L2 grammar learners have, the less effect
of instructional methods on learner performance will be obtained.
1) Participants with less prior knowledge on the target form will benefit from both explicit
explanation and visual enhancement more than those with more prior knowledge.
2) The benefit of explicit explanation will decrease as participants’ levels of expertise in the
target L2 grammar increase.
3) The benefit of an animated pedagogical agent and a flashing arrow will decrease as
57
participants’ levels of expertise in the target L2 grammar increase.
Descriptive statistics (e.g., means, standard deviations) will be used to estimate the
participants’ performances on the pretest and the post test, and MANOVA will be run to examine
the relationship between learner prior knowledge and various outcome measures (i.e., interest,
attention, time, and performance tests). If the MANOVA is significant, individual ANOVA will
be performed.
Hypothesis 4 – There will be no causal relationship between learners’ levels of interest in
instructional program they interacted with and the level of learner performance measured by the
comprehension and production tests on the target grammar.
Descriptive statistics (e.g., means, standard deviations) will be used to estimate the
participants’ interest levels and performances on post tests, and multiple regression analysis will
be used to examine the relationship between learners’ interest and their performances on the
posttests.
Hypothesis 5 - Participants’ levels of attention directed to the target L2 grammar during the
processing of the learning material will have significant effect on the level of learner
performance measured by the comprehension and production tests on the target grammar.
Descriptive statistics (e.g., means, standard deviations) will be used to estimate the
participants’ attention and performances on post tests, and multiple regression analysis will be
used to examine the relationship between learners’ attention and their performances on the
posttests.
Hypothesis 6 – The flashing arrow will require less time and mental effort from participants than
the animated pedagogical agent in achieving the same level of learning performance, when both
media are used to deliver the visual enhancement treatment.
58
Descriptive statistics (e.g., means, standard deviations) will be run to estimate the
measures of each group’s outcomes on the amount time and mental effort required to master the
target form. A T-test will be performed to test the significance of differences on these measures
between the two groups.
Hypothesis 7 – There will be a significant positive correlation between the level of learner prior
knowledge of the target grammar and the level of cognitive efficiency.
Descriptive statistics (e.g., means, standard deviations) will be used to estimate the
measures on the levels of learner prior knowledge and the levels of cognitive efficiency. A
Pearson product correlation will be used to assess the relationship between the learner prior
knowledge and the levels of cognitive efficiency.
59
References
Ainley, M., Hidi, S., & Berndorff, D. (2002). Interest, and the psychological processes that
mediate their relationship. Journal of Educational Psychology, 94(3), 545-561.
Alanen, R. (1995). Input enhancement and rule presentation in second language acquisition.
Attention and awareness in foreign language learning. Schmidt, R. Honolulu, University
of Hawaii Press, Tech. Rep. No. 9, pp. 259-302.
Andre, E., Rist, T., & Muller, J. (1999). Employing AI methods to control the behavior of
animated interface agents. Applied Artificial Intelligence, 13, 415-448.
Atkinson, R. K. (2002). Optimizing learning from examples using animated pedagogical agents.
Journal of Educational Psychology, 94(2), 416-427.
Baddeley, A. D. (1992). Working memory. Science, 255, 556-559.
Beentjes, J. W. J. (1989). Learning from television and books: A Dutch replication study based
on Salomon's Model. Educational Technology Research and Development, 37(2), 47-58.
Bobis, J., Sweller, J., & Cooper, M. (1993). Cognitive load effects in a primary-school geometry
task. Learning and Instruction, 3, 1-21.
Bradshaw, J. M. (1997). Software agents. Cambridge, MA, MIT Press.
Casali, J. G. W., Wierwille, W. W., & Cordes, R. E. (1983). A comparison of rating scale,
secondary task, physiological and primary-task workload estimation techniques in a
simulated flight task emphasizing communication load. Human Factors, 25(6), 623-641.
Cennamo, K. S. (1992). Students' perceptions of the ease of learning from computers and
interactive video: An exploratory study. Journal of Educational System, 21, 251-263.
Cennamo, K. S. (1993). Learning from video, Factors influencing learners' perceptions and
60
invested mental effort. Educational Technology Research and Development, 41(3), 33-45.
Cennamo, K. S. (1996). The effect of relevance on mental effort. ERIC document reproduction
service No. ED397 783.
Chandler, P., & Sweller, J. (1991). Cognitive load theory and the format of introduction.
Cognition and Instruction, 8, 293-332.
Clark, R. E. (1983). Reconsidering research on learning from media. Review of Educational
Research, 53(4), 445-459.
Clark, R. E. (1994a). Media and method. Educational Technology Research & Development,
42(3), 7-10.
Clark, R. E. (1994b). Media will never influence learning. Educational Technology Research &
Development, 42(2), 21-29.
Clark, R. E. (1998). Cognitive efficiency research on media: A rejoinder to Cobb. An
unpublished manuscript.
Clark, R. E. (1999b). Yin and Yang Cognitive Motivational Process Operating in Multimedia
Learning Environment. In J. Van Merrienboer (Ed.), Cognition and Multimedia Design.
Herleen, Netherlands: Open University Press.
Clark, R. E. (2001). Learning from media: Arguments, analysis and evidence. Greenwich, CT:
Information Age Publishers.
Clark, R. E. (In press, 2003), Research on web-based learning: A half-full glass. In Bruning, R.,
Horn, C. and PytlikZillig, L. (In Press) Web-Based Learning: Where do we know? Where
do we go? Greenwich, CT: Information Age Publishers.
Clark, R. E., & Choi, S. (2003, October). Six Suggestions for the Design of Experiments on the
Effects of Animated Pedagogical Agents. Paper presented at the Annual Convention of
61
the Association of Educational Computer & Technology, Anaheim, CA.
Cobb, T. (1997). Cognitive Efficiency, Toward a revised theory of media. Educational
Technology Research & Development, 45(4), 21-35.
Craig, S. D., Gholson, B., & Driscoll, D. M. (2002). Animated pedagogical agents in multimedia
educational environments: Effects of agent properties, picture features and redundancy.
Journal of Educational Psychology, 94(2), 428-434.
Curran, T., & Keele, S. W. (1993). Attentional and nonattentional forms of sequence learning.
Journal of Experimental Psychology, Learning, Memory and Cognition, 19, 189-202.
Dalgarno, B. (2001). Interpretations of constructivism and consequences for computer assisted
learning. British Journal of Educational Technology, 32(2), 183-194.
DeKeyser, R. (1995). Learning second language grammar rules: An experiment with a miniature
linguistic system. Studies in Second Language Acquisition, 17(3), 379-410.
DeKeyser, R. (1998). Beyond focus on form: Cognitive perspectives on learning and practicing
second language grammar. In C. Doughty and J. Williams (Eds.), Focus on Form in
Classroom Second Language Acquisition (pp. 42-63), Cambridge, MA: Cambridge
University Press.
Dehn, D. M. & van Mulken, S. (2000). The impact of animated interface agents: a review of
empirical research.
International Journal of Human-Computer Studies, 52, 1-22.
Doughty, C. (1991). Second language does make a difference, Evidence from an empirical study
of SL relativization. Studies in Second Language Acquisition, 13, 431-469.
Dulany, D. E. (1991). Conscious representations and thought systems. Advances in social
cognition (Vol. 4). R. Wuer, J., & T. Srull. Hillsdale, NJ, Erlbaum.
Dweck, C. S. (1989). Motivation. In A. R. G. Lesgold (Ed.), Foundations for a Psychology of
62
Education. Hillsdale, New Jersey: Lawrence Erlbaum Associates.
Ellis, N., Ed. (1994). Implicit and explicit learning of languages. London, Academic Press.
Erickson, T. (1997). Designing agents as if people mattered. In J. M. Bradshaw (Ed.), Software
Agents (pp. 79-96). Menlo Park, CA: MIT Press.
Fisher, S. L., & Ford, J. K. (1998). Differential effects of learner effort and goal orientation on
two learning outcomes. Personnel Psychology, 51(2), 397-420.
Flad, J. A. (2002). The effects of increasing cognitive load on self-report and dual-task measures
of mental effort during problem solving. Unpublished doctoral dissertation, University of
Southern California. Los Angeles, California.
Garner, R., Brown, R., Sanders, S., & Menke, D. (1992). Seductive details and learning from text.
In K. A. Renninger, S. Hidi, & A. Krapp (Eds.), The role of interest in learning and
development (pp. 239-254). Hillsdale, NJ: Erlbaum.
Gimino, A. E. (2000). Factors that influence students' investment of mental effort in academic
tasks: A validation and exploratory study. Unpublished doctoral dissertation, University
of Southern California. Los Angeles, California.
Harp, S. F., & Mayer, R. E. (1997). The role of interest in learning from scientific text and
illustrations: On the distinction between emotional interest and cognitive interest.
Journal of Educational Psychology, 89(1), 92-102.
Harp, S. F., & Mayer, R. E. (1998). How seductive details do their damage, A theory of cognitive
interest in science learning. Journal of Educational Psychology, 90, 414-434.
Hidi, S., & Anderson, V. (1992). Situational interest and its impact on reading and expository
writing. In K., A. Renninger, S. Hidi, and A. Krapp (Eds.), The Role of Interest in
Learning and Development (pp. 215 – 238), Hillsdale, NJ: Lawrence Erlbaum
63
Associates, Publishers.
Izumi, S. (2002). Output, input enhancement, and the noticing hypothesis: An experimental study
on ESL relativization. Studies in Second Language Acquisition, 24, 541-577.
Jeung, H., Chandler, P., & Sweller, J. (1997). The role of visual indicators in dual sensory mode
instruction. Educational Psychology, 17(3), 329-343.
Johnson, W. L., Ricke. J. W., & Lester, J. C. (2000). Animated pedagogical agents, Face-to-face
interaction in interactive learning environments. International Journal of Artificial
Intelligence in Education, 11, 47-78.
Jourdenais, R., Ota, M., Stauffer, S., Boyson, B., & Doughty, C. (1995). Does textual
enhancement promote noticing? A think-aloud protocol analysis. Attention & Awareness
in foreign language learning. Schmidt, R. Honolulu, University of Hawai'i Press.
Kalyuga, S., Chandler, P., & Sweller, J. (1998). Levels of expertise and instructional design.
Human Factors, 40(1), 1-17.
Kalyuga, S., Chandler, P., & Sweller, J (1999). Managing split-attention and redundancy in
multimedia instruction. Applied Cognitive Psychology, 13, 351-371.
Kalyuga, S., Chandler, P., & Sweller, J (2000). Incorporating learner experience into the design
of multimedia instruction. Journal of Educational Psychology, 92(1), 126-136.
Kimmer, H., & Deek, F. (1996). Instructional technology: A tool or a panacea? Journal of
Science Education and Technology, 5(1), 87-92.
Kotovosky, K. H., J. R., & Simon, H. A. (1985). Why are some problems hard? Evidence from
Tower of Hanoi. Cognitive Psychology, 17, 248-294.
Leow, R. (1998). Toward operationalizing the process of attention in SLA: Evidence for Tomlin
and Villa's (1994) fine-grained analysis of attention. Applied Psycholinguistics, 19, 133-
64
159.
Lester, J. C., Converse, S. A., Kahler, S. E., Barlow, S. T., Stone, B. A., & Bhogal, R. S. (1997,
March). The persona effect, Affective impact of animated pedagogical agents.
Proceedings of Computer-Human Interaction '97 (pp. 359-366), Atlanta.
Lester, J. C., Converse, S. A., Stone, B., Khaler, S., & Barlow, T. (1997). Animated Pedagogical
Agents and Problem-Solving Effectiveness: A Large-Scale Empirical Evaluation.
Proceedings of the 8th World Conference on Artificial Intelligence in Education (pp. 2330), Kobe, Japan.
Lester, J. C., Voerman, J., Towns, S., & Callaway, C. (1997). Cosmo, A life-like animated
pedagogical agent with deictic believability. IJCAI '97 Workshop on Animated Interface
Agents, Making them intelligent, Nagoya, Japan.
Levin, H. M., & McEwan, P. M. (2001). Cost-effectiveness analysis: Methods and applications
(2nd ed.). Thousand oaks, CA: Sage.
Long, M. H. (1983). Does second language instruction make a difference? TESOL Quarterly,
17(3), 359-382.
Lowe, J. (2002). Computer-based education: Is it a panacea? Journal of Research on Technology
in Education, 34(2), 163-171.
Mayer, R. E., & Moreno, R. (1998). A Split-attention effect in multimedia learning: Evidence for
dual processing systems in working memory. Journal of Educational Psychology, 90,
312-320.
Mayer, R. E., Heiser, J., & Lonn, S. (2001). Cognitive constraints on multimedia learning: When
presenting more materials results in less understanding. Journal of Educational
Psychology, 93(1), 187-198.
65
Mayer, R. E., Moreno, R., Boire, M., & Vagge, S. (1999). Maximizing constructivist learning
from multimedia communications by minimizing cognitive load. Journal of Educational
Psychology, 91(4), 638-643.
Moreno, R., & Mayer, R. E. (1999). Cognitive principles of multimedia learning: The role of
modality and contiguity. Journal of Educational Psychology, 91, 358-368.
Moreno, R., & Mayer, R. E. (2000a). A coherence effect in multimedia learning: The case for
minimizing irrelevant sounds in the design of multimedia instructional messages.
Journal of Educational Psychology, 97, 117-125.
Moreno, R., & Mayer, R. E. (2000b). A learner-centered approach to multimedia explanations:
Deriving instructional design principles from cognitive theory. Interactive Multimedia
Electronic Journal of Computer-Enhanced Learning, 2(2). Retrieved April 17, 2002,
from http://imej.wfu.edu/articles/2000/2/05/index.asp.
Moreno, R., Mayer, R. E., & Lester, J. C. (2000). Life-like pedagogical agents in constructivist
multimedia environments, Cognitive consequences of their interaction. The World
Conference on Educational Multimedia, Hypermedia, and Telecommunications (EDMEDIA), Montreal, Canada.
Moreno, R., Mayer, R. E., Spires, H. A., & Lester, J. C. (2001). The case for social agency in
computer-based teaching: Do students learn more deeply when they interact with
animated pedagogical agents? Cognition and Instruction, 19(2), 177-213.
Mousavi, S. Y., Low R., & Sweller, J. (1995). Reducing cognitive load by mixing auditory and
visual presentation modes. Journal of Educational Psychology, 87(2), 319-334.
Nass, C., & Steuer, J. (1993). Anthropomorphism, agency, and thopoeia: Computers as social
actors. Human Communication Research, 19(4), 504-527.
66
Norris, J. M., & Ortega, L. (2000). Effectiveness of L2 instruction: A research synthesis and
quantitative meta-analysis. Language Learning, 50(3), 417-528.
Reber, A. S. (1976). Implicit learning of synthetic languages: The role of instructional set.
Journal of Experimental Psychology, Human Learning and Memory, 2, 88-94.
Reber, A. S. (1989). Implicit learning and tacit knowledge. Journal of Experimental Psychology,
General, 118, 219-235.
Richards, J. C., & Rodgers, T. S. (1998). Approaches and methods in language teaching, A
description and analysis. New York, Cambridge University Press.
Reeves, B., & Nass, C. (1996). The Media Equation: How people treat computers, television,
and new media like real people and places. New York: Cambridge University Press.
Robinson, P. (1995). Review article, Attention, memory, and the noticing hypothesis. Language
Learning, 45, 283-331.
Robinson, P. (1996). Learning simple and complex second language rules under implicit,
incidental, rule search, and instructed conditions. Studies in Second Language
Acquisition, 18, 27-68.
Robinson, P. (1997a). Generalizability and automaticity of second language learning under
implicit, incidental, enhanced, and instructed conditions. Studies in Second Language
Acquisition, 19, 223-247.
Robinson, P. (1997b). Individual differences and the fundamental similarity of implicit and
explicit adult second language learning. Language Learning, 47, 45-99.
Rosa, E., & O'Neill, M. (1999). Explicitness, intake, and the issue of awareness. Studies in
Second Language Acquisition, 21, 511-556.
Salomon, G. (1983). The differential investment of mental effort in learning from different
67
sources. Educational Psychologist, 18(1), 42-50.
Salomon, G. (1984). Television is easy and print is tough: The differential investment of mental
effort in learning as a function of perceptions and attributions. Journal of Educational
Psychology, 76(4), 647-658.
Salomon, G., & Leigh, T. (1984). Predispositions about learning from print and television.
Journal of Communication, 34, 119-125.
Sampson, D., Karagiannidis, C., & Kinshuk (2002). "Personalised learning: Educationla,
technological and standardisation perspective." Interactive Educational Multimedia, 4,
24-39.
Schmidt, R. (1990). The role of consciousness in second language learning. Applied Linguistics,
11, 206-226.
Schmidt, R. (1995). Consciousness and foreign language learning: A tutorial on the role of
attention and awareness in learning. In R. Schmidt (Ed.), Attention and awareness in
foreign language learning. Honolulu, Hawaii, University of Hawaii Press.
Schmidt, R. (2001). Attention. In P. Robinson (Ed.), Cognition and second language instruction
(pp. 3-32). New York: Cambridge University Press.
Shiffrin, R., & Schneider, W. (1977). Controlled and automatic human information processing, II.
Perceptual learning, automatic attending, and a general theory. Psychological Review, 84,
127-190.
Spada, N. (1997). Form-focused instruction and second language acquisition: A review of
classroom and laboratory research. Language Teaching, 29, 1-15.
Sweller, J. (1999). Instructional design in technical areas. Camberwell, Australia: ACER Press.
Sweller, J., & Chandler, P. (1994). Why some material is difficult to learn. Cognition and
68
Instruction, 12, 185-233.
Sweller, J., Cooper, G. A., Tierney, P., & Cooper, M. (1990). Cognitive load and selective
attention as factors in the structuring of technical material. Journal of Experimental
Psychology, General, 119, 176-192.
Sweller, J., van Marrienboer, J. J. G., & Paas, F. G. W. C. (1998). Cognitive architecture and
instructional design. Educational Psychology Review, 10, 251-296.
Tomlin, R., & Villa, V. (1994). Attention in cognitive science and second language acquisition.
Studies in Second Language Acquisition, 16, 183-203.
van Gerven, P. W. M., Paas, F. G. W. C., & Schmidt, H. G. (2000). Cognitive load theory and the
acquisition of complex cognitive skills in the elderly: Towards an integrative framework.
Educational Gerontology, 26, 503-521.
van Mulken, S., Andre, E., & Muller, J. (1998). The person effect: How substantial is it? In H.
Johnson, L. Nigay & C. Roast (Eds.), People and Computers XIII: Proceedings of
HCI’98, pp. 53-66. Berlin: Springer.
Ward, M., & Sweller, J. (1990). Structuring effective worked examples. Cognition and
Instruction, 7, 1-39.
Wertsch, J., & Bivens, J. A. (1992). The social origins of individual mental functioning,
Alternatives and perspectives. The Quarterly Newsletter of the Laboratory of
Comparative Human Cognition, 14, 35-44.
Wierwille, W. W., Rahimi, M., & Casali, J. G. (1985). Evaluation of 16 measures of mental
workload using a simulated flight task emphasizing mediational activity. Human Factors,
27, 489-502.
White, J. (1998). Getting the learners' attention: A typographical input enhancement study. In C.
69
Doughty and J. White (Eds.), Focus on form in classroom second language acquisition
(pp.85-113). New York, NY: Cambridge University Press.
Williams, J. (1999). Memory, attention, and inductive learning. Studies in Second Language
Acquisition, 21, 1-48.