Trust and Semantic attacks
Ponnurangam Kumaraguru (PK)
Usable, Privacy, and Security
Mar 17, 2008
CMU Usable Privacy and Security Laboratory
http://cups.cs.cmu.edu/
Who am I?

Ph.D. candidate in the Computation,
Organizations, and Society program in the
School of Computer Science

Research interests - Privacy, Security,
Trust, Human Computer Interaction, and
Learning Science
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
2
Outline

Trust

Semantic attacks - Phishing

User education

Learning science

Evaluating embedded training

Ongoing work

Conclusion
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
3
What is trust?

No single definition

Depends on the
situation and the
problem

Many models
developed

Very few models
evaluated
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
4
Trust in literature

Economics (how trust affects transactions)
• Reputation

Marketing (how to build trust)
• Persuasion

HCI (what affects trust)
• Design

Psychology (positive theory)
• Intimacy
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
5
Trust Models

Positive antecedents
• Benevolence
• Comprehensive
information
• Credibility
• Familiarity
• Good feedback
• Propensity
• Reliability
• Usability
• Willingness to
transact
•…

Negative antecedents
• Risk
• Transaction cost
• Uncertainty
•…
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
6
How do users make decisions?

Interview design, 25 participants (11 experts and 14 - non-experts)

Measured the strategies and decision
process of the users in online situations

Results
• Non-experts wanted advice to help them make
better trust decisions
• Non-experts used significantly fewer
meaningful signals compared to experts
P. Kumaraguru, A. Acquisti, and L. Cranor. Trust modeling for online transactions: A phishing scenario.
In Privacy Security Trust, Oct 30 - Nov 1, 2006, Ontario, Canada.
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
7
Expert model
Unknown states
Not deliberate states
States that
affect decision
Misleading
signals
Signals
Meaningful
signals
States that
affect well-being
Missed
signals
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
8
Non- expert model
Unknown states
Not deliberate states
States that
affect decision
Misleading
signals
States that
affect well-being
Signals
Meaningful
signals
Missed
signals
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
9
Outline

Trust

Semantic attacks - Phishing

User education

Learning science

Evaluating embedded training

Ongoing work

Conclusion
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
10
Security Attacks: Waves

Physical: attack the computers, wires and
electronics
 E.g. physically cutting the network cable

Syntactic: attack operating logic of the
computers and networks
 E.g. buffer overflows, DDoS

Semantic: attack the user not the
computers
 E.g. Phishing
http://www.schneier.com/essay-035.html
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
11
Semantic Attacks

“Target the way we, as humans, assign
meaning to content.”

System and mental model
http://groups.csail.mit.edu/uid/projects/phishing/proposal.pdf
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
12
An email that we get
Features in the email
Subject: eBay: Urgent Notification From Billing Department
Features in the email
We regret to inform you that you eBay account could be
suspended if you don’t update your account information.
Features in the email
https://signin.ebay.com/ws/eBayISAPI.dll?SignIn&sid=veri
fy&co_partnerid=2&sidteid=0
Website to collect information
http://www.kusi.org/hcr/eBay/ws23/eBayISAPI.htm
What is phishing?
Phishing is “a broadly launched social
engineering attack in which an
electronic identity is misrepresented in
an attempt to trick individuals into
revealing personal credentials that can
be used fraudulently against them.”
Financial Services Technology Consortium. Understanding and countering the phishing threat: A financial service
industry perspective. 2005.
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
18
Phishing Attack Life Cycle
Fraud & Abuse
Setup
Source:http://www.coopercain.com/User%20Data/A%20Leisurely%20Lunch%20Time%20Phishing%20Trip-show.ppt
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
19
A few statistics on phishing

73 million US adults received more than 50
phishing emails each in the year 2005

Gartner in 2006 found 30% users changed online
banking behavior because of attacks like phishing

Gartner in 2006 predicted $2.8 billion loss due to
phishing in that year
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
20
Why phishing is a hard problem?

Semantic attacks take advantage of the
way humans interact with computers

Phishing is one type of semantic attack

Phishers make use of the trust that users
have on legitimate organizations
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
21
Three strategies for usable
privacy and security

Invisible strategy
• Regulatory solution
• Detecting and deleting the emails

User interface based
• Toolbars

Training users
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
22
Our Multi-Pronged Approach

Human side
• Interviews to understand decision-making
• PhishGuru embedded training
• Anti-Phishing Phil game
• Understanding effectiveness of browser
warnings

Computer side
• PILFER email anti-phishing filter
• CANTINA web anti-phishing algorithm
Automate where possible, support where necessary
Outline

Trust

Semantic attacks - Phishing

User education

Learning science

Evaluating embedded training

Ongoing work

Conclusion
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
24
Why user education is hard?

Security is a secondary task

Users not motivated to taking time for
education

Non-existence of an effective method
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
25
To address the open questions

Embedded training methodology
• Make the training part of primary task
• Create motivation among users

Learning science
• Principles for designing training interventions
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
26
Approaches for training

Posting articles
• FTC,…

Phishing IQ tests
• Mail Frontier, …

Classroom training
(Robila et al.)

Sending security notices
http://www.ftc.gov/bcp/conline/pubs/alerts/phishingalrt.htm
http://www.sonicwall.com/phishing/
http://pages.ebay.com/education/spooftutorial/
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
27
Security notices
• How to spot an email
• How to report spoof email
• Five ways to protect yourself from identity theft
Outline

Trust

Semantic attacks - Phishing

User education

Learning science

Evaluating embedded training

Ongoing work

Conclusion
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
29
Why learning science?

Research on how people gain knowledge
and learn new skills

ACT-R theory of cognition and learning
• Declarative knowledge (knowing that)
• Procedural knowledge (knowing how)

Learning science principles
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
30
Learning science principles
 Learning-by-doing
• More practice better performance
 Story-based agent
• Using agents in a story-based content
enhances user learning
 Immediate feedback
• Feedback during learning phase results in
efficient learning
Clark, R.C., and Mayer, R.E. E-Learning and the science of instruction: proven guidelines for consumers and designers
of multimedia learning. John Wiley & Sons, Inc., USA, 2002.
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
31
Learning science principles
 Conceptual-procedural
• Presenting procedural materials in between
conceptual materials helps better learning
 Contiguity
• Learning increases when words and pictures
are presented contiguously than isolated
 Personalization
• Using conversational style rather than formal
style enhances learning
Clark, R.C., and Mayer, R.E. E-Learning and the science of instruction: proven guidelines for consumers and designers
of multimedia learning. John Wiley & Sons, Inc., USA, 2002.
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
32
Outline

Trust

Semantic attacks - Phishing

User education

Learning science

Evaluating embedded training

Ongoing work

Conclusion
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
33
Design constraints

People don’t proactively read the training
materials on the web

People can learn from web-based training
materials, if only we could get people to
read them! (Kumaraguru et al.)
P. Kumaraguru, S. Sheng, A. Acquisti, L. Cranor, and J. Hong. Teaching Johnny Not to Fall for Phish.
Tech. rep., Cranegie Mellon University, 2007. http://www.cylab.cmu.edu/files/cmucylab07003.pdf.
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
34
Embedded training

We know people fall for phishing emails

So make the training available through the
phishing emails

Training materials are presented when the
users actually fall for phishing emails

Makes training part of primary task

Creates motivation among users

Applies learning-by-doing and immediate
feedback principle
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
35
Embedded training example
Subject: Revision to Your Amazon.com Information
Embedded training example
Subject: Revision to Your Amazon.com Information
Please login and enter your information
http://www.amazon.com/exec/obidos/sign-in.html
Comic strip intervention
Design rationale

What to show in the intervention?

When to show the intervention?

Analyzed instructions from most popular
websites

Paper and HTML prototypes, 7 users each

Lessons learned
• Two designs
• Present the training materials when users click
on the link
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
39
Study 1: Evaluation of interventions

H1: Security notices are an ineffective
medium for training users

H2: Users make better decisions when
trained by embedded methodology
compared to security notices
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
40
Study design

Think aloud study

Role play as Bobby Smith, 19 emails including
2 interventions, and 4 phishing emails

Three conditions: security notices, text /
graphics intervention, comic strip intervention

10 non-expert participants in each condition,
30 total
P. Kumaraguru, Y. Rhee, A. Acquisti, L. Cranor, J. Hong, and E. Nunge. Protecting People from
Phishing: The Design and Evaluation of an Embedded Training Email System. CyLab Technical
Report. CMU-CyLab-06-017, 2006. http://www.cylab.cmu.edu/default.aspx?id=2253 [to be presented
at CHI 2007]
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
41
Intervention #1 - Security notices
• How to spot an email
• How to report spoof email
• Five ways to protect yourself from identity theft
Intervention # 2 - Comic strip
Intervention # 2 - Comic strip
Applies personalization and story based principle
Presents declarative knowledge
Intervention # 2 - Comic strip
Applies personalization principle
Intervention # 2 - Comic strip
Applies contiguity principle
Intervention # 2 - Comic strip
Applies contiguity and conceptual-procedural principle
Presents procedural knowledge
Intervention # 3 - Text / graphics
User involvement
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
49
Legitimate
Phish
Training
Spam
User study - results

We treated clicking on link to be falling for
phishing

93% of the users who clicked went ahead
and gave personal information
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
51
User study - results
100
Percentage of user s who
clicked on the link
90
80
70
60
50
40
30
20
10
0
3: Phish
5:
Training
7: Legit
11:
Training
13: Legit
14:
Phish-N
16:
Phish-N
17:
Phish
Emails which had a link in them
Notices
Text / Graphics
Comic
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
52
User study - results

Significant difference between security
notices and the comic strip group
(p-value < 0.05)

Significant difference between the comic
and the text / graphics group
(p-value < 0.05)
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
53
Lessons learned

H1: Security notices are an ineffective
medium for training users

H2: Users make better decision when
trained by embedded methodology
compared to security notices
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
54
Open questions

Previous studies measured only knowledge
gain

Users have specific knowledge than
generalized knowledge (Downs et al.)

What about knowledge retention and
transfer?
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
55
Knowledge retention and transfer

Knowledge retention (KR)
• The ability to apply the knowledge gained after
a time period

Knowledge transfer (KT)
• The ability to transfer the knowledge gained
from one situation to another situation
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
56
Study design

Setup
• Think aloud study
• Role play as Bobby Smith, business administrator
• Respond to Bobby’s email

Experiment
• Part 1: 33 emails and one intervention
• Part 2 (after 7 days): 16 emails and no intervention

Conditions
•
•
•
•
Control: no intervention
Suspicion: an email from a friend
Non-embedded: intervention in the email
Embedded: intervention after clicking on link
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
57
Sample of emails from study
Email type
Sender
Subject information
Legitimate-no-link
Brandy Anderson
Booking hotel rooms
for visitors
Legitimate-link
Joseph Dicosta
Please check PayPal
balance
Phishing-no-account
Wells Fargo
Update your bank
information!
Phishing-account
eBay
Reactivate your eBay
account
Spam
Eddie Arredondo
Fw: Re: You will want
this job
Intervention
Amazon
Revision to your
Amazon.com
information
Comic strip intervention
Hypotheses

H1: Participants in the embedded condition
learn more effectively than participants in
the non-embedded condition, suspicion
condition, and the control condition

H2: Participants in the embedded condition
retain more knowledge about how to
avoid phishing attacks than participants in
the non-embedded condition, suspicion
condition, and the control condition
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
60
Hypotheses

H3: Participants in the embedded condition
transfer more knowledge about how to
avoid phishing attacks than participants in
the non-embedded condition, suspicion
condition, and the control conditions
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
61
Study results

We treated clicking on link to be falling for
phishing

89% of the users who clicked went ahead
and gave personal information
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
62
Results - Phishing account emails
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
63
Results - Legitimate link emails
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
64
Measuring retention

Training on Amazon.com account revision
phish

Testing a week later on Citibank account
revision phish

Significant difference between embedded
and other groups (p < 0.01)

“I remember reading last time that thing
[training material] said not click and give
personal information.”
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
65
Measuring transfer

Training on Amazon.com account revision
phish

Testing a week later on eBay account
reactivation phish

Significant difference between embedded
and other groups (p < 0.01)

“PhishGuru said not to click on links and
give personal information, so will not do it, I
will delete this email.”
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
66
A few observations

“I was more motivated to read the training
materials since it was presented after me
falling for the attack.”

“Thank you PhishGuru, I will remember that
[the 5 instructions given in the training
material].”

“This [image in the email] looks like some
spam.”
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
67
Outline

Trust

Semantic attacks - Phishing

User education

Learning science

Evaluating embedded training

Ongoing work

Conclusion
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
68
Ongoing work

Test the system in real-world
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
69
Conclusion
Educating users about security can be a
reality rather than just a myth
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
70
Collect homework
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
71
Acknowledgements

Members of Supporting Trust Decision
research group

Members of CUPS lab
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
72
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
73
CMU Usable Privacy and Security
Laboratory
http://cups.cs.cmu.edu/
Learning-by-doing principle

Production rules are acquired and
strengthened through practice

More practice better performance

Story-centered curriculum

Cognitive tutors
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
75
Immediate feedback principle

Feedback during knowledge acquisition
phase results in efficient learning

Corrects behavior

Avoids floundering

LISP tutors

“yes” or “no” or detailed
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
76
Conceptual-Procedural principle

A concept is a mental representation or
prototype of objects or ideas

A procedure is a series of clearly defined
steps

Presenting procedural materials in between
conceptual materials helps better learning

Studies
• Mathematics
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
77
Contiguity principle

Learning increases when words and
pictures are presented contiguously rather
than isolated from one another

Human learning process - creating
meaningful relation between pictures and
words

Studies
• Vehicle braking system
• Geometry cognitive tutor
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
78
Personalization principle

Using conversational style rather than
formal style enhances learning

To use “I,” “we,” “me,” “my,” “you,” and
“your” in the instructional materials

Studies
• Process of lightning formation
• Mathematics
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
79
Story-based agent principle

Characters who help in guiding the users
through the learning process

Using agents in a story-based content
enhances user learning

Stories simulate cognitive process

Experiments - Herman
• CMU Usable Privacy and Security Laboratory • http://www.cs.cmu.edu/~ponguru
80