Understanding frustration in e

Frustration in E-learning
Managing frustration and encouraging
learner resiliency with affective
Ken Rafanan
Brian Tobin
Contact: [email protected]
December 3, 2002
In order to keep up with market demands and with the competition, a company needs to train new
employees well and keep current employees up-to-date with the current policies. Traditionally,
corporate training occurs in a lecture or workshop setting at company gatherings or on retreats.
The biggest drawback is that such training can be costly since large companies would require
numerous trainers to do the same programs with many groups of employees.
E-learning has been emerging as an alternative to the traditional corporate training. Corporate Elearning involves giving employees access to any number of electronic information sources,
including computerized readings, hypertext, information databases, message boards, and email
Regardless of the quality of an e-learning program, if a user loses interest in learning from the
program or gives up on using it the program will cease to benefit that learner as well as the
company. User frustration with the software with the software or even with computers in general
could lead to discontinued usage or effectiveness of the program.
We spent 70 hours investigating current research and propose that designers employ the
following design principles for maintaining the effectiveness of E-learning programs:
Motivate users to manage their frustration with the program
Convince users that they are in control of their emotional state
Organization of this chapter
This chapter will explore the relationship between e-learning and frustration. First, we will
explain the issues surrounding frustration and e-learning. Next, we will clearly state our
guidelines and rationale. Then, we will use scenarios and storyboards to give examples of how an
e-learning program may deal with user frustration. Next, we will address variation and tough
decisions that arise when dealing with frustration in e-learning. Then, we will explain what we
don’t yet know about frustration in e-learning and give some references and suggestions for how
Rafanan & Tobin
[email protected]
Frustration in E-Learning
to learn more about dealing with frustration in e-learning. Finally, we will wrap up this chapter by
summarizing our guidelines and recommendations.
Understanding frustration in e-learning
Frustration detracts from learning by lessening users’ attention, retention, learning, thinking
creatively, normal reasoning capacities, and increases the likelihood of frustration with that
source in the future. Currently, software provides, at best, passive emotional regulation, like
pulling attention away from the source of frustration or allowing the user to vent. However,
passive emotional regulation does not encourage people to overcome the source of frustrationrather it simply alleviates the current emotional state.
An effective way to help users overcome or deal with frustration is for the software to provide
active listening by giving sincere, non-judgmental, sympathetic or empathetic feedback to a
frustrated individual. Klein, Moon, and Picard (2002) found that even rudimentary active
listening responses from software can significantly decrease users’ frustration and cause longer
usage and more satisfaction with frustrating software.
Target users
Our research and recommendations will benefit adult employees of large and small businesses
who participate in e-learning programs. Their participation in such training and learning programs
is likely mandatory.
How frustration fits into the larger scheme of things
Research by Nass and Reeves showed that people anthropomorphize computers and form
relationships with media. Wenger (1998) stated that positive relationships between participants in
learning environments can enhance the resulting learning. Thus, we assert that the relationships
that users form with e-learning software can influence positively users’ learning. If users become
extremely frustrated with the software, then this poor relationship with the software will detract
from their learning.
What are the subsets of designing a program to deal with user frustration
Designing an e-learning environment is especially difficult because not only have to worry about
users not just as users but also as learners. Designers of e-learning programs can alleviate user
frustration in two major ways:
1. Finding and removing frustrating tasks and elements from the program
2. Helping users how to better deal with potentially frustrating tasks and concepts
The first of these recommendations should be a familiar principle of good user interface design.
The second recommendation is more difficult to deal with. Sometimes it is the challenging and
potentially frustrating content that offers the most learning opportunities and what is challenging
and enjoyable for some users may be simply frustrating for others.
Our goal is to provide designers with guidelines on how to create the most effective e-learning
environments. We firmly believe that the most effective learning can occur only when the
software helps learners deal with potential frustrations.
Rafanan & Tobin
[email protected]
Frustration in E-Learning
Guidelines for dealing with frustration
We suggest adhering to two main guidelines for helping users deal with possibly frustrating
1. Teach your learners general frustration management techniques
2. Human learners are emotional beings so design your software accordingly
Teach your learners general frustration management techniques.
We believe that the most important lesson your e-learning program can teach your employees is
how to manage their frustration better. If your learners understand themselves more and know the
ways they can work through frustrating situations, then they will be less likely to become
frustrated and will use your e-learning software better, longer, and will learn more.
There are many different
suggestions and proposed
methods for how individuals
can deal with frustration. We
have included some sample
frustration management
techniques in this sidebar but
recommend that you develop a
targeted curriculum for your
employees that provides them
with recommendations and
gives them opportunities to
practice dealing with frustration.
Frustration Management Techniques
Applicable to E-Learning
From “Controlling anger- before it controls you”,
pamphlet by American Psychological Association
 Relax and refocus
 Change the way you’re thinking about the problem
 Change the way you think about your situation
 Give yourself a break
From “Dealing with Frustration”, by David Gershaw
 Keep trying: repeated efforts, successes, and
failures may help you achieve your goal
 Consult with others on how to achieve your goal
 Decide if there are alternate methods to achieving
your goal
 Determine whether there are similar or alternate
goals that are more achievable
Human learners are emotional beings so design your software accordingly
Use human-human interaction as your model
Giving your e-learning system some key human characteristics will go a long way in helping your
learners to become comfortable with using the system and maintaining a positive disposition
towards the software even when frustration is encountered. A number of persuasion techniques
should be employed in building the relationship with the user.
Make a good first impression
Build a positive relationship with the learner from her first interaction with the program. As with
any human-human interaction, laying out the relationship in advance and explaining the role of
Rafanan & Tobin
[email protected]
Frustration in E-Learning
each participant will help the relationship continue running smoothly. The e-learning program
should state what it will do and what the user can do. The system should build credibility as a
learning tool by letting the learner know how well it’s been designed. For example, it could say
“I’ll present the lessons in a way that’s proven effective for most people. If you don’t understand
something then I can give you a little more help”. The user will eventually learn that some of the
content or some aspects of the system will be frustrating so allow the system to gain the users’
trust by confiding to the user that it is not flawless and will not always be easy to use. In addition,
computer users are not immune to flattery. If the program tells learners how intelligent they are,
how quickly they pick up on things, or other flattering things the user may enjoy working on the
program more.
Pay attention to users’ feelings and
Even simple attention to users’
feelings can build rapport with the
system or encourage users to work
through frustration. Users’
frustration may be predicted by
identifying commonly frustrating
situations or modules in the system,
monitoring time spent or mouse
clicks, or through more complicated
methods of measuring stress. Likely
points of frustration could be
greeted with examples of active
listening by the computer. Active
listening could have many
components. We suggest the
Poll learner on current emotional state
How are you feeling about your progress?
1. I’m doing fine. Thanks for asking!
2. I’m having a little trouble but will manage.
3. I’m pretty frustrated.
4. I’m totally frustrated with this program!!!
System’s Reply
1. User: I’m doing fine. Thanks for asking!
System: That’s great! I’m glad we’re working well
2. User: I’m having a little trouble but will manage.
System: Alright. Your work ethic is impressive!
Keep at it.
3. User: I’m pretty frustrated.
System: I’m sorry. If you keep at it maybe you’ll get
it. Would you like to review your frustration
management techniques? See the bulletin board?
Send feedback?
4. User: I’m totally frustrated with this program!!!
System: I’m really sorry! Do you want to go to the
Bulletin Board? Just move on? Would you like to
give the programmers feedback?
Actively solicit information about
the user’s emotional state, especially
Asking the user how he or she is
feeling about progress is a simple
way to obtain users’ frustration
levels and to determine if an
intervention is necessary.
Respond to users’ frustration with sympathy and empathy
Even the simplest examples of affective computing have been proven effective at lessening users’
frustration levels and increasing.
Predict what support a learner might need
By providing support before a learner experiences frustration, your system validates the learner’s
emotional state and lessens her propensity to become frustrated.
Rafanan & Tobin
[email protected]
Frustration in E-Learning
Examples of applying these guidelines
Scenario 1: Junko
Junko, 25 year old new hire from Japan
knows little about the company and its
processes and procedures. She wants to learn
everything and to learn it well. She is both
intrinsically motivated because she wants to
be a good employee and is extrinsically
motivated because she must complete the elearning program to become a manager.
Junko has been using the e-learning system
for a few hours and is about to begin the
section about budgeting. Since the software
designers have identified the budgeting
module as a likely source of frustration for
users, the software warns Junko that the
module is particularly difficult and asks if she
would like to review the frustration
management techniques she learned earlier.
She chooses to review these lessons and this
review helps her relax and stay focused on the
Rafanan & Tobin
[email protected]
Frustration in E-Learning
Scenario 2: Steve
Steve, a 35 year old white male – has been a
manager for six years. Steve thinks that he knows
everything he needs for his job managing sales
people in his region, and he doesn’t appreciate
being forced into the mandatory training. Steve
wants to get through the training as fast as possible
so that he can concentrate on what he feels are his
higher job priorities. Because of this disposition,
Steve is likely to suffer frustration if he can’t work
through a module on the first try.
Steve works on a lesson going over the company’s
personnel policies dealing underperforming
employees. Thinking that he is well-aware of the
policies, Steve barely skims the material. Due to a
recent change in labor law, the company’s “first
response” policy now requires managers to provide
a written request for improvement as well as a 360
evaluation soliciting feedback from the target
In the module post-test, which determines whether
the learner may move on, a question about this
company policy has been made pass/fail – users
answering incorrectly will be forced to work
through the module a second time. In the post-test,
Steve receives a score of 8 of 10, but he fails the
test because he missed the question about the
company’s first response policy. Steve becomes
frustrated – he believes he knows the material well
enough as evidenced by an 80% score.
The frustration response system knows that a failure on the test can lead to learner frustration.
Before Steve has a chance to move beyond frustration into anger and suffer the corresponding
decrease in learning ability, the FRS polls Steve on his level of frustration. His response of 4 out
of 4 (indicating a very high level of frustration) causes the system to provide options of using a
Bulletin Board, skipping the section or providing feedback to the programmers as ways to work
through this frustration.
Rafanan & Tobin
[email protected]
Frustration in E-Learning
Steve chooses the bulletin board because he wants
to know why he isn’t passing with an 80% grade.
On the bulletin board he asks if anyone else has
had the same problem. Another user had read her
test results a little more carefully; she tells Steve
that some of the questions require correct
responses for a passing grade. Steve looks back at
the detailed test results and sees that he had
indeed missed one of the pass/fail questions. He is
still not very happy, but he isn’t as frustrated
because he better understands his situation.
Steve returns to the module paying closer
attention this time.
Issues in dealing with frustration
Attempting to build a relationship between the learner and a learning system can be very
beneficial. However, a designer needs to be aware that a poor relationship may be worse than
none at all. As is clear from people’s reactions to Microsoft Word’s Clippy agent, overly frequent
interventions or misdiagnosing problems can lead to greater frustration than no intervention at all.
We must also guard against eliminating all sources of frustration as these may be rich learning
opportunities. One user’s engaging challenge can be another’s source of immense frustration.
Rafanan & Tobin
[email protected]
Frustration in E-Learning
What we don’t yet know about frustration in e-learning
There are not yet many studies investigating what frustrates users in e-learning and what design
principles designers can use to alleviate or help users deal with frustration. We make the best
recommendations we can with the current body of research but propose a few future studies to
1. particularly frustrating situations, tasks, and concepts specific to e-learning environments
2. reliable methods to monitor users’ frustration while using a computer
3. additional effective techniques for reducing frustration within an e-learning environment
How to learn more about frustration in e-learning
Frustration and Computing
Professor Rosalind Picard is leading the Affective Computing Research Group at the MIT Media
Lab (http://affect.media.mit.edu/AC_affect.html). They state:
we think mechanisms of emotion will be needed to build "human-centered" machines,
which are able to respond intelligently and sensitively to the complex and unpredictable
situations common to human-computer interaction
Their research will prove valuable when determining how frustration management is possible
within e-learning.
Users’ Relationships with Computers
We recommend reading The Media Equation for ideas on the social relationships users can form
with computers. Learners will form some sort of relationship with an e-learning program so
designers of these systems should decide what kind of relationships they want to foster.
E-learning has the potential to greatly augment corporate training. Within any curriculum, the
teacher/designer must endeavor to keep users motivated, but we must expect some learner
frustration and the resulting decrease in motivation. E-learning designers, then, should be
prepared to address this frustration using our guidelines.
We developed these guidelines using grounded theories and current research on how to deal with
learner frustration in computing. We believe that e-learning systems should ensure that their
learners are aware of how to deal with frustration and pay attention to and intervene in situations
when learners need extra help dealing with frustration. Only when designers remember that users’
emotions and feelings are an integral part of how they use and learn from an e-learning system
will e-learning fulfill its promise and truly earn its place among proven learning tools.
Rafanan & Tobin
[email protected]
Frustration in E-Learning
References and resources
Controlling anger- before it controls you. Retrieved December 2, 2002 from PsychNET
Gershaw, D. (1997). Dealing with Frustration. Retrieved from A Line on Life
Klein, J., Moon, Y., & Picard, R.W. (2002). This computer responds to user frustration: Theory,
design, and results. Interacting with computers, 14, pp. 119-139.
Lawson, R. (1965). Frustration: The development of a scientific concept. Macmillan, NY.
Reeves, B. & Nass, C. (1996). The Media Equation: How People Treat Computers, Television,
and New Media Like Real People and Places. Cambridge: Cambridge University Press.
Scheirer, Fernandez, Klein, & Picard. (2002). Frustrating the user on purpose: a step toward
building an affective computer. Interacting with computers, 14, pp. 93-118.
Wenger, E. (1998). Communities of Practice: Learning, Meaning and Identity. Cambridge:
Cambridge University Press.
Rafanan & Tobin
[email protected]
Related flashcards


42 cards

System administration

65 cards


61 cards

Corporate governance

32 cards

Create Flashcards