Applied Ergonomics 86 (2020) 103084
Contents lists available at ScienceDirect
Applied Ergonomics
journal homepage: http://www.elsevier.com/locate/apergo
Email phishing and signal detection: How persuasion principles and
personality influence response patterns and accuracy
Patrick Lawson *, Carl J. Pearson, Aaron Crowson, Christopher B. Mayhorn
Department of Psychology, North Carolina State University, Raleigh, NC, USA
A R T I C L E I N F O
A B S T R A C T
Keywords:
Phishing
Cybersecurity
Susceptibility
Signal detection
Persuasion principle
Personality
Phishing is a social engineering tactic where a malicious actor impersonates a trustworthy third party with the
intention of tricking the user into divulging sensitive information. Previous social engineering research in a realworld setting has shown an interaction between the personality of the target and the persuasion principle used.
This study investigated whether this interaction is present in the realm of email phishing. Additionally, a signal
detection theory framework was used to evaluate how the various persuasion principles influence accuracy,
sensitivity (d’), and response criterion placement. A personality inventory and an email identification task
(phishing or legitimate) were used. These data support previous findings that high extroversion is predictive of
increased susceptibility to phishing attacks. The various persuasions principles elicited diverse response crite­
rions and sensitivities, though all investigated persuasion principles resulted in a liberal decision criterion, except
one. These findings are interpreted and discussed.
1. Introduction
Phishing is a social engineering tactic designed to trick users into
divulging sensitive personal information, such as one’s social security or
bank account numbers, through impersonation of a trustworthy third
party (Jagatic et al., 2007). Here we focus on untargeted phishing at­
tacks (i.e. not spear-fishing attacks where a specific person is targeted)
distributed via email, and examine factors potentially related to in­
dividuals’ susceptibility to such attacks.
There are various strategies that can be employed to persuade a
target to divulge their sensitive information. Cialdini identified six broad
persuasion principles (Cialdini, 1987). Such principles have been found
to be used in email settings (Ferreira et al., 2015; Ferreira and Teles,
2019; Parsons et al., 2019). One study found four of these principles to
be more applicable, and more frequently employed, in phishing emails
than the other two persuasion principles (Akbar, 2014). Furthermore,
these same four persuasion principles have been found to be increasing
in volume over time in phishing emails (Zielinska et al., 2016). Given the
high baseline use and pattern of increasing use, we will concern our­
selves primarily with these persuasion principles. The four persuasion
principles and brief descriptions are as follows (Cialdini, 1987):
� Commitment/consistency: the concept of completing an action you
previously initiated.
� Liking: trust due to a prior interaction or familiarity, such as for a
largely recognizable brand.
� Authority: an authority figure mandating an action, with conse­
quences for failing to comply.
� Scarcity: a short and specific time frame to complete an action.
The excluded principles are Social Proof (imitating others’ behavior),
and Reciprocity (returning a favor) (Cialdini, 1987).
While content is important, user characteristics may also influence
how emails are perceived, impacting email phishing susceptibility. For
instance, younger individuals have been shown to be more susceptible
than older individuals (Kumaraguru et al., 2009; Sheng et al., 2010).
Experiential factors also play a role, such that those who have previously
received phishing identification training are less susceptible (Mayhorn
and Nyeste, 2012; Sheng et al., 2007), as are those who self-report high
knowledge of technology (Sheng et al., 2010). In addition, the person­
ality profile of the victim plays a role in the likelihood of being phished.
High distrust of others is positively correlated with accuracy in identi­
fying phishing emails (Welk et al., 2015). Generally, high extroversion is
found to be one of the personality traits most predictive of increased
phishing susceptibility as demonstrated by Workman (2008) through
* Corresponding author. Department of Psychology, North Carolina State University, 700A Poe Hall, Campus Box 7650, Raleigh, NC, 27695-7650, USA.
E-mail address: palawson@ncsu.edu (P. Lawson).
https://doi.org/10.1016/j.apergo.2020.103084
Received 13 December 2018; Received in revised form 25 February 2020; Accepted 25 February 2020
0003-6870/© 2020 Elsevier Ltd. All rights reserved.
P. Lawson et al.
Applied Ergonomics 86 (2020) 103084
their use of a field study. In their study, Workman (2008) gave a ques­
tionnaire designed to assess levels of commitment, trust, obedience to
authority, and reactance/resistance to employees of a large service or­
ganization based in the United States. Employees were then sent
phishing attacks in the form of emails designed to get users to click on
Web page URLs, or to download attached executable files. They found
that people with high affective commitment as well as high normative
commitment were more likely to fall prey to attacks, both of which
relate significantly to extroversion (Erdheim et al., 2006). Similarly,
high normative commitment was significantly related to agreeableness,
which is another personality trait associated with an increased phishing
susceptibility (Erdheim et al., 2006; Parrish et al., 2009).
The previously mentioned traits agreeableness and extroversion -in
addition to neuroticism, openness, and conscientiousness-comprise the
five personality constructs of the Five-Factor Model of personality,
colloquially known as the ‘Big Five’ (Costa and McCrae, 1992a; Tupes
and Christal, 1961). These traits have been shown to be stable over time,
and universally identifiable regardless of language, race, culture, or
gender (Costa and McCrae, 1992b).
Notably, when looking at social engineering in the real world (i.e.
not online), interaction effects between the persuasion principle used
and the personality of the target have been demonstrated (Uebelacker
and Quiel, 2014). Through the use of a comprehensive literature review
of each of the five personality traits Uebalacker and Quiel (2014) found,
for example, that extroverted individuals are especially susceptible to
the liking and scarcity persuasion principles, while agreeable in­
dividuals are especially susceptible to the authority principle, among
other such interactions (Uebelacker and Quiel, 2014).
Taken together, these findings indicate that 1) many different
persuasion principles exist and are utilized in phishing emails, 2) po­
tential victims’ personality profiles are related to phishing susceptibility,
and 3) that efficacy of real-world social engineering is modulated by an
interaction between the persuasion principle used and the victim’s per­
sonality profile. This paper investigates whether this interaction be­
tween the persuasion principle and the user’s personality exists within
the realm of email phishing attacks. We hypothesize that many of the
interaction effects Uebelacker and Quiel (2014) theorized in real-world
social engineering will also be present in email phishing attacks. This
prediction is based on work demonstrating similar uses of persuasion
principles within real-world and email modalities (Ferreira et al., 2015;
Ferreira and Teles, 2019; Parsons et al., 2019). Specifically, it is pre­
dicted that agreeableness will be predictive of susceptibility to author­
ity, and extroversion will be predictive of susceptibility to liking as well
as scarcity. Both of these hypotheses were demonstrated in Uebelacker
and Quiel’s (2014) real-world social engineering literature review. In
addition, it is hypothesized that high extroversion will be predictive of
overarching susceptibility to phishing emails.
In making classification judgements regarding emails (phishing or
legitimate), it is relevant to consider the confidence of such judgements.
Overconfidence in one’s abilities often lead to errors in judgements
(Sulistyawati et al., 2011), including misidentifying phishing emails
(Wang et al., 2016). Sulistyawati et al. (2011) suggest that over­
confidence causes inadequate analysis of a situation before a decision is
made, leading to an error. With this in mind, we expect to replicate the
findings from Wang et al. (2016), such that overconfidence is expected
to similarly contribute to errors in the present email identification task.
However, new to this study is the analysis of how overconfidence varies
by the persuasion principle of each email.
Finally, the email classification task used here lends itself well to a
signal detection theory analysis, as at least one other study has explored
(Canfield et al., 2016). Signal detection theory allows for the identifi­
cation of the participants’ decision criterion, which is a measure of
response bias and whether certain types of emails are inherently trusted
or distrusted (Green and Swets (1966). A conservative decision criterion
would reflect trust in a certain persuasion principle, while a liberal de­
cision criterion would reflect distrust of that persuasion principle. If a
decision criterion is placed conservatively it will result in a higher
instance of misses than false alarms, while liberal decision criterions will
result in higher instances of false alarms than misses.
It is possible that the various persuasion principles may evoke
different response patterns; some principles or combinations of princi­
ples may generally be trusted and assumed to be legitimate emails, while
others may be distrusted and assumed to be phishing attempts. Such bias
is observable through the rates of false alarms and correct rejections (as
well as hits and misses) incurred by each persuasion principle or com­
bination of principles. Such a signal detection analysis may give insight
into the underlying causes of susceptibility to phishing emails. While
signal detection theory has been used in previous phishing research,
such studies tended to treat all phishing emails as roughly homogenous,
without considering the specific content or persuasion principle of each
email (Canfield et al., 2016). It is hypothesized that emails utilizing the
authority and scarcity principles will be inherently distrusted, resulting
in liberal decision criterions. Finally, a comparison of the d’ values will
demonstrate how sensitivity varies for each of the persuasion principles
(or combinations of persuasion principles) investigated here.
2. Method
2.1. Participants
One hundred and two participants (mean age 19.3 years old; SD ¼
2.8) were recruited from an undergraduate psychology course at a large
Southeastern university in the United States and given class credit for
participation. Fifty-four participants were female. This research com­
plied with the American Psychological Association Code of Ethics and
was approved by the Institutional Review Board at North Carolina State
University (IRB Protocol #7794). All participants received and elec­
tronically signed an informed consent form prior to the start of the study.
All participants were at least 18 years old.
2.2. Materials
A total of 90 emails were used in this experiment. All 45 legitimate
emails were drawn from the experimenters’ personal (non-academic)
email addresses. These emails were selected from a larger group because
they met two primary criteria: 1) they attempted to persuade the
recipient to perform some action, and 2) they clearly contained at least
one of the four Cialdini persuasion principles of interest. Because these
emails were drawn from the researchers’ personal rather than their ac­
ademic email addresses, there is little reason to expect that the partici­
pants had increased likelihoods of having received these same emails.
All 45 phishing emails were drawn from a corpus of confirmed phishing
attacks compiled from three prominent universities (Zielinska et al.,
2016). None of these universities were the university at which the pre­
sent study was conducted, making it unlikely participants would have
received the same phishing emails selected for use as stimuli. Only
sensitive identifying information was removed, such as the recipient’s
name and email address; otherwise, the emails were unaltered. The
emails were coded according to all persuasion principles utilized. Three
raters coded these emails, and there was an 87% agreement between
raters (Zielinska et al., 2016). This analysis of phishing emails by divi­
sion into categories according to all persuasion principles present was
proposed and demonstrated to be of value by Ferreira et al. (2015). As
mentioned above, four of Cialdini’s persuasion principles were consid­
ered: commitment/consistency (C), liking (L), authority (A), and scar­
city (S).
After considering the prevalence of each principle and its likelihood
of being combined with other principles, nine groups of persuasion
principles (or combinations of principles) were derived: A, A/C, A/L, A/
S, C, C/L, L, S, and Super (Su). The Super category was defined as using
at least three of the four core Cialdini principles assessed in this study (A,
C, L, S). The number of emails in each category can be seen in Table 1.
2
P. Lawson et al.
Applied Ergonomics 86 (2020) 103084
3. Results
Table 1
Number of emails per group.
Persuasion Principle(s)
Legitimate Emails
Phishing Emails
Authority
Authority & Commitment/Consistency
Authority & Liking
Authority & Scarcity
Commitment/Consistency
Liking & Commitment/Consistency
Liking
Scarcity
Super (3þ)
Total
2
5
5
5
5
6
6
5
6
45
5
5
5
5
5
5
5
5
5
45
3.1. Overview
All variables analysed were approximately normally distributed.
First, Pearson correlations between the primary variables of interest
(‘Big Five’, impulse control, trust, overall confidence, legitimate confi­
dence, phishing confidence, overall accuracy, legitimate accuracy, and
phishing accuracy) were computed to broadly assess covariance. Next,
accuracy and confidence differences between the phishing and legiti­
mate groups were assessed. The phishing and legitimate groups were
then each subdivided according to the persuasion principle(s) used, to
investigate accuracy for each persuasion principle individually. Multiple
regressions were used to investigate the contribution of each personality
trait to the observed email identification accuracies. Finally, t-tests were
conducted to investigate how confidence contributes to the accuracy of
email identification for each persuasion principle.
Due to an inability to identify five legitimate emails exclusively using
the authority principle in a natural context, and a desire to maintain an
equal total number of legitimate and phishing emails, three legitimate
groups comprised six emails rather than five. Below are two examples of
phishing emails, one utilizing the liking principle, and the other utilizing
the authority & scarcity principles (Figs. 1 and 2). Additionally, there
are two examples of legitimate emails utilizing the same persuasion
principles (Figs. 3 and 4).
The trust subsection of the IPIP NEO PI-R was used to assess trust
(Costa and McCrae, 1992c). The impulse control subsection of the IPIP
AB5C Facets Abbreviated Scale was used to assess impulse control. The
‘Big Five’ personality traits (neuroticism, extroversion, openness,
agreeableness, conscientiousness) were assessed with the NEO–FFI–3,
which is a shorter version of the NEO-PI-3 with only 60 items that
measures the five domains of personality (12 items per domain) (Costa
and McCrae, 1992c).
3.2. Legitimate vs. Phishing accuracy
A paired samples t-test was conducted to determine if there were
differences between the accuracy of responses for phishing and legiti­
mate emails. Phishing accuracy (M ¼ 0.66, SD ¼ 0.47) was found to be
significantly greater than legitimate accuracy (M ¼ 0.62, SD ¼ 0.48), t
(4589) ¼ 4.15, p < .001. That is, participants were more likely to
correctly label a phishing email as phishing than to correctly label a
legitimate email as legitimate.
3.3. Correlations
2.3. Procedure
Pearson correlations were used to create a correlation matrix of the
primary variables of interest. A Benjamini-Hochberg correction was
used to control for False Discovery Rates (FDR). Benjamini-Hochberg pvalues are reported in this section, with FDR set to .05 (Benjamini and
Hochberg, 1995). These primary variables of interest included the seven
personality measures (the ‘Big Five’, impulse control, and trust), overall
confidence, legitimate confidence, phishing confidence, overall accu­
racy, legitimate accuracy, and phishing accuracy.
There are a few findings of note from this matrix. First, impulse
control was positively correlated with phishing detection accuracy (r ¼
0.29, p ¼ .024). That is, as impulse control increased, phishing detection
accuracy also tended to increase. Impulse control was also correlated
with three personality principles. Agreeableness was positively
Personality measures were collected from all participants. The
experiment was conducted entirely online, hosted on Qualtrics survey
software.
To investigate the hypothesized interaction between the user’s per­
sonality and the persuasion principle utilized, an email identification
task was used. Participants were asked to identify whether 90 emails
were phishing attempts or legitimate emails. They then rated the con­
fidence associated with their choice from 0 to 100% certainty. Upon
completion of the experiment participants were thanked for their
participation and awarded class credit.
Fig. 1. Example of a phishing email utilizing the liking persuasion principle.
3
P. Lawson et al.
Applied Ergonomics 86 (2020) 103084
Fig. 2. Example of a phishing email utilizing the authority & scarcity persuasion principle.
Fig. 3. Example of a legitimate email utilizing the liking persuasion principle.
correlated with impulse control (r ¼ 0.45, p < .001), while extroversion
and neuroticism were both negatively correlated with impulse control,
(r ¼ 0.346, p < .001) and (r ¼ 0.31, p ¼ .019), respectively. Overall
accuracy was positively correlated with agreeableness (r ¼ 0.28, p ¼
.038). Phishing detection accuracy was negatively correlated with
extroversion (r ¼ 0.36, p < .001).
portion of this paper.
Looking at the graph of legitimate email accuracies (Fig. 5), we see
that the emails making use of the liking persuasion principle were on the
upper end of the identification accuracy spectrum. That is, participants
were unlikely to mislabel these legitimate emails as phishing attempts
(again, liking emails include a largely recognizable brand). On the other
end of the spectrum, legitimate emails utilizing both authority & scarcity
were likely to be incorrectly labelled as phishing attempts, with more
than half of participants (54%) incorrectly labelling such emails as
phishing attempts.
Looking at the graph of phishing emails (Fig. 6), we see a very
different trend. Phishing emails making use of the liking principle
showed low identification accuracies; that is, participants demonstrated
a susceptibility to these emails. Participants failed to identify these
phishing emails in 53% of trials. In contrast, phishing emails utilizing
authority & scarcity showed high identification accuracies, and were
likely to be correctly identified as phishing emails. As reported earlier,
accuracy for phishing emails (M ¼ 0.66, SD ¼ 0.47) was found to be
significantly greater than accuracy for legitimate accuracy (M ¼ 0.62,
3.4. Accuracy by persuasion principle
Next, identification accuracy for the various persuasion principles
(or combinations of principles) was investigated. This was done without
considering the personality profile of the responder. These results can be
seen in Figs. 5 and 6. Notably, no corrections were applied to these data
to account for a general liberal or conservative stance. This is inten­
tional, as the ratio of phishing emails to legitimate emails was one to
one. As such, there was no incentive for the participant to bias responses
in one direction or another. Any differential accuracy between phishing
and legitimate emails is thus notable, as it is evidence of overarching
bias. This will be explored in greater depth in the signal detection theory
4
P. Lawson et al.
Applied Ergonomics 86 (2020) 103084
Fig. 4. Example of a legitimate email utilizing the authority & scarcity persuasion principle.
Fig. 5. Accuracy identifying various legitimate emails. Error bars indicate 95%
confidence interval.
Fig. 6. Accuracy identifying various phishing emails. Error bars indicate 95%
confidence interval.
SD ¼ 0.48), t(4589) ¼ 4.15, p < .001. This indicates a tendency to
categorize emails as phishing attempts, even though the number of
legitimate and phishing emails in the experiment was equal.
Considering both the legitimate and phishing accuracies together, it
appears that emails making use of both the authority & scarcity prin­
ciples were likely to arouse suspicion, regardless of whether they were
legitimate or phishing emails. Conversely, emails utilizing the liking
principle appeared unlikely to arouse suspicion, regardless of whether
they were legitimate or phishing emails.
3.5. Multiple regressions
To assess the interaction of the various personality traits with the
persuasion principle utilized, multiple linear regressions were con­
ducted. In each of the following regressions the predictor variables
entered in the model were impulse control, trust, neuroticism, extro­
version, openness, agreeableness, and conscientiousness (i.e. all seven of
the personality measures). All multiple regressions were conducted
using the Enter method.
Inputting all the listed predictor variables, a significant model was
found for phishing accuracy F(7,101) ¼ 3.37, p ¼ .003, R2 ¼ 0.20. This
5
P. Lawson et al.
Applied Ergonomics 86 (2020) 103084
model explained 20% of the observed variance. As can be seen in
Table 2, it was found that high extroversion was a significant predictor
of decreased phishing accuracy (β ¼ 0.33, p ¼ .007). Notably, a sig­
nificant model was not found with legitimate accuracy or overall ac­
curacy as the outcome variable, and none of the predictor variables in
these models reached significance, even at the p ¼ .05 threshold.
Next, regressions were conducted with each of the nine individual
persuasion principles (or combinations of principles) as the outcome
variable, using the same predictor variables as above (i.e. all seven
personality measures). Nine separate linear regressions were thus con­
ducted, one for each persuasion principle (or combination of principles).
We first looked at the phishing emails.
As can be seen in Table 3, extroversion was found to be predictive of
decreased detection of phishing attacks utilizing: authority & commit­
ment/consistency persuasion (β ¼ 0.31, p ¼ .015), authority & liking
persuasion (β ¼ 0.29, p ¼ .021), commitment/consistency persuasion
(β ¼ 0.30, p ¼ .017), and liking persuasion (β ¼ 0.28, p ¼ .024). In the
five cases where extroversion was not significantly predictive of
increased susceptibility to a persuasion principle, the results were
trending in the direction of increased susceptibility. Conscientiousness
was found to be predictive of increased detection of phishing attacks
utilizing super persuasion (β ¼ 0.24, p ¼ .031).
The same steps were then used in the analysis of the legitimate
emails. Here, trust was found to be predictive of increased correct
identification of legitimate emails utilizing commitment/consistency
persuasion (β ¼ 0.25, p ¼ .033). Openness was found to be predictive of
increased correct identification of legitimate emails utilizing super
persuasion (β ¼ 0.23, p ¼ .035). These results can be found in Table 4.
be seen in the authority and scarcity principle (legitimate M ¼ 0.70, SD
¼ 0.24, phishing M ¼ 0.58, SD ¼ 0.25) t(356) ¼ 3.79, p < .001).
Notably, a reverse relationship can be seen when looking at the liking
principle. Participants were more confident in their responses when
incorrectly identifying a phishing liking email (M ¼ 0.68, SD ¼ 0.22) as
opposed to when they incorrectly identified a legitimate liking email (M
¼ 0.59, SD ¼ 0.23), t(449) ¼ -4.30, p < .001.
Finally, an analysis of overconfidence was conducted for each
persuasion principle, once for legitimate emails and then again for
phishing emails. Overconfidence was calculated using the method
described in Wang et al. (2016). Confidence (subjective probability of
accuracy) was subtracted from accuracy (actual probability of accu­
racy), yielding a single measure of confidence. Overconfidence would be
indicated by positive values, where true performance was worse than
expected based on subjective judgements. A series of paired samples
t-tests with Bonferroni corrections were then used to compare these
means. These results can be seen in Table 7.
A significant overconfidence was found for legitimate emails utiliz­
ing the authority & scarcity principles (Accuracy M ¼ 0.46, SD ¼ 0.19,
Confidence M ¼ 0.68, SD ¼ 0.14) t(101) ¼ 10.09, p < .001, and
legitimate emails using the authority & commitment/consistency prin­
ciple (Accuracy M ¼ 0.55, SD ¼ 0.22, Confidence M ¼ 0.64, SD ¼ 0.14) t
(101) ¼ 3.45, p < .001. This means that for legitimate emails utilizing
the authority & scarcity persuasion principle or the authority &
commitment/consistency principle, participants were more confident in
their responses than they were accurate.
A significant overconfidence was found for phishing emails utilizing
the liking persuasion principle (Accuracy M ¼ 0.47, SD ¼ 0.22, Confi­
dence M ¼ 0.67, SD ¼ 0.14), t(101) ¼ 8.04, p < .001, and phishing
emails using the liking & commitment/consistency principle (Accuracy
M ¼ 0.53, SD ¼ 0.27, Confidence M ¼ 0.64, SD ¼ 0.16) t(101) ¼ 3.53,
p < .001. This means that for phishing emails using either the liking
principle or the liking & commitment/consistency principle participants
were more confident in their responses than they were accurate.
Additionally, two instances of underconfidence were observed. A
significant underconfidence was found for phishing emails using the
authority principle (Accuracy M ¼ 0.82, SD ¼ 0.17, Confidence M ¼
0.67, SD ¼ 0.17), t(101) ¼ 7.24, p < .001, as well as for phishing emails
utilizing the authority & scarcity principle (Accuracy M ¼ 0.84, SD ¼
0.21, Confidence M ¼ 0.67, SD ¼ 0.17) t(101) ¼ 7.18, p < .001. This
means that for phishing emails using either the authority principle or the
authority & scarcity principle participants were less confident in their
responses than they were accurate.
3.6. Legitimate vs. phishing confidence
Independent sample t-tests with Bonferroni corrections were con­
ducted to determine if there were differences in confidence ratings be­
tween correctly and incorrectly identified emails. These results can be
seen in Table 5. In all cases of significance participants had more con­
fidence in their responses when they were correct as opposed to when
they were incorrect.
Another set of independent samples t-tests with Bonferroni correc­
tions were conducted to further asses if there were confidence differ­
ences between incorrectly identified phishing emails as opposed to
incorrectly identified legitimate emails. Comparisons were made across
persuasion principles. The same analysis was utilized for correctly
identified legitimate emails as opposed to correctly identified phishing
emails. These results can be found in Table 6. No significant differences
were found between correctly identified phishing emails and correctly
identified legitimate emails for any persuasion principle. However,
when looking at the incorrectly identified emails, significant differences
were found for the authority, authority & scarcity, and liking principles.
Specifically, when a participant incorrectly identified an email using the
authority principle they were more confident in their response when the
email was legitimate (M ¼ 0.66, SD ¼ 0.21) than when the email was
phishing (M ¼ 0.56, SD ¼ 0.22), t(164) ¼ 2.95, p ¼ 0.004. The same can
3.7. Signal detection theory
The last major analyses conducted approaches the problem of email
classification (as either phishing or legitimate) from a signal detection
theory perspective. To achieve this, it was assumed that the difficulty or
degree to which an email arouses suspicion of being a phishing attempt
is normally distributed for both legitimate and phishing emails. Desig­
nating phishing emails as the signal to be identified, responses were
classified as hits (phishing emails identified as phishing), misses
(phishing emails identified as legitimate), false alarms (legitimate
emails identified as phishing), and correct rejections (legitimate emails
identified as legitimate).
Each of the 9 persuasion principles (or combinations of persuasion
principles) were analysed independently. The d’ values, false alarm
rates, miss rates, and decision criterion status (liberal or conservative)
may be found in Table 8 below.
The d’ values ranged from 0.412 to 1.23; these relatively small d’
values reflect the difficulty of this email identification task. Decision
criterions were placed liberally for 8 of the 9 persuasion principles (or
combinations of persuasion principles), indicating that participants
preferred to err on the side of caution, generating more false alarms than
misses. Nonetheless, misses were relatively high (especially considering
Table 2
Beta coefficients for overall, phishing, and legitimate identification accuracies.
Personality Characteristic
Impulse Control
Trust
Neuroticism
Extroversion
Openness
Agreeableness
R2
Accuracy
Overall
Phishing
Legitimate
0.15
0.09
0.10
0.15
0.04
0.22
0.16
0.11
0.05
0.02
0.33**
0.10
0.19
0.20
0.07
0.07
0.11
0.15
0.15
0.08
0.08
* Indicates significance at the p ¼ .05 threshold. ** Indicates significance at the
p ¼ .01 threshold.
6
P. Lawson et al.
Applied Ergonomics 86 (2020) 103084
Table 3
Beta coefficients for phishing identification accuracies of each persuasion principle(s).
Personality
Characteristic
Persuasion Principle(s)
Authority
Authority &
Commitment/
Consistency
Authority &
Liking
Authority &
Scarcity
Commitment/
Consistency
Liking
Liking &
Commitment/
Consistency
Scarcity
Super
(3þ)
Impulse Control
Trust
Neuroticism
Extroversion
Openness
Agreeableness
Conscientiousness
R2
0.03
0.04
0.08
0.15
0.14
0.25
0.09
0.09
0.06
0.05
0.17
0.31*
0.14
0.08
0.10
0.14
0.07
0.05
0.12
0.29*
0.01
0.15
0.10
0.14
0.24
0.03
0.05
0.01
0.03
0.06
0.10
0.06
0.08
0.05
0.09
0.30*
0.13
0.01
0.030
0.13
0
0.20
0.03
0.28*
0.16
0.20
0.15
0.19
0.03
0.11
0.11
0.16
0.00
0.03
0.09
0.04
0.06
0.02
0.15
0.19
0.05
0.14
0.11
0.10
0.27
0.16
0.10
0.04
0.07
0.12
0.24*
0.17
* Indicates significance at the p ¼ .05 threshold. ** Indicates significance at the p ¼ .01 threshold.
Table 4
Beta coefficients for legitimate identification accuracies of each persuasion principle(s).
Personality
Characteristic
Persuasion Principle(s)
Authority
Authority &
Commitment/
Consistency
Authority &
Liking
Authority &
Scarcity
Commitment/
Consistency
Liking
Liking &
Commitment/
Consistency
Scarcity
Super
(3þ)
Impulse Control
Trust
Neuroticism
Extroversion
Openness
Agreeableness
Conscientiousness
R2
0.18
0.03
0.12
0.06
0.06
0.02
0.02
0.03
0.02
0.14
0.10
0.03
0.00
0.03
0.02
0.02
0.07
0.08
0.03
0.24
0.16
0.11
0.14
0.08
0.06
0.03
0.12
0.08
0.15
0.03
0.10
0.04
0.02
0.25*
0.02
0.15
0.12
0.02
0.13
0.11
0.08
0.07
0.18
0.20
0.11
0.05
0.03
0.07
0.10
0.00
0.04
0.03
0.20
0.06
0.06
0.05
0.04
0.02
0.17
0.16
0.04
0.18
0.15
0.09
0.04
0.04
0.08
0.09
0.23*
0.05
0.17
0.08
* Indicates significance at the p ¼ .05 threshold. ** Indicates significance at the p ¼ .01 threshold.
Table 5
Confidence ratings for correct and incorrect responses according to email type
and persuasion principle.
Persuasion
Principle
Authority
Authority &
Commitment/
Consistency
Authority & Liking
Authority &
Scarcity
Commitment/
Consistency
Liking
Liking &
Commitment/
Consistency
Scarcity
Super (3þ)
Overall
Legitimate Emails
Correct
0.70
0.67
Table 6
Comparing legitimate and phishing response confidence for correct and incor­
rect responses.
Phishing Emails
>
>
Incorrect
0.66
0.62
Correct
0.67
0.68
>*
>*
Incorrect
0.56
0.57
0.67
0.67
>**
<
0.60
0.70
0.68
0.69
>**
>**
0.61
0.58
0.68
>**
0.62
0.72
>**
0.62
0.69
0.69
>**
>**
0.59
0.62
0.67
0.6.6
<
>
0.68
0.62
0.67
0.68
0.68
>
>
>**
0.65
0.66
0.64
0.68
0.65
0.68
>
>
>**
0.64
0.6.2
0.62
* Indicates significance at the p ¼ .05 threshold. ** Indicates significance at the
p ¼ .01 threshold.
Persuasion
Principle
Incorrect Responses
Authority
Authority &
Commitment/
Consistency
Authority &
Liking
Authority &
Scarcity
Commitment/
Consistency
Liking
Liking &
Commitment/
Consistency
Scarcity
Super (3þ)
Overall
0.66
0.62
Legitimate
Correct Responses
Phishing
Legitimate
>**
>
0.56
0.57
0.70
0.67
>
<
Phishing
0.70
0.68
0.56
<
0.61
0.67
<
0.68
0.70
>**
0.58
0.67
<
0.69
0.62
>
0.62
0.68
<
0.72
0.58
0.62
<**
>
0.68
0.62
0.69
0.69
>
>
0.66
0.66
0.65
0.66
0.64
>
>
>
0.64
0.62
0.62
0.67
0.68
0.68
<
>
<
0.68
0.65
0.68
* Indicates significance at the p ¼ .01 threshold.
the cost of misses in a phishing context), partially owing to the relatively
low d’ values elicited. The most liberal decision criterion was observed
for the combined authority & scarcity principle, resulting in a 54.1%
false alarm rate and a 16.1% miss rate. The authority & scarcity signal
detection graph may be found in Fig. 7.
The only persuasion principle with a conservative decision criterion
was liking, which aims to appeal to the interests of the recipient. The
liking persuasion principle resulted in a 53.3% miss rate, and a 29.2%
false alarm rate. The low d’ value results in a high level of ambiguity
regarding whether emails using this principle are phishing or legitimate.
The liking signal detection graph may be found above, in Fig. 8. Signal
detection graphs for all other persuasion principles may be found in the
supplemental section.
4. Discussion
Extroversion was found to be a strong predictor of overarching sus­
ceptibility to phishing attacks, corroborating previous research
(Workman, 2008). It was the only factor significantly predictive of
overall phishing identification ability and achieved significance at the p
¼ .01 threshold. Extroversion was also significantly associated with
increased susceptibility to four persuasion principles: liking (as was
7
P. Lawson et al.
Applied Ergonomics 86 (2020) 103084
Table 7
Overconfidence for legitimate and phishing emails.
Persuasion Principle
Legitimate Emails
Authority
Authority & Commitment/Consistency
Authority & Liking
Authority & Scarcity
Commitment/Consistency
Liking
Liking & Commitment/Consistency
Scarcity
Super (3þ)
Overall
Phishing Emails
Accuracy
Confidence
Overconfidence
Accuracy
Confidence
Overconfidence
0.62
0.55
0.65
0.46
0.61
0.71
0.67
0.63
0.64
0.62
0.68
0.64
0.68
0.68
0.66
0.66
0.67
0.66
0.67
0.66
0.06
0.09*
0.03
0.22*
0.05
0.05
0
0.03
0.03
0.04
0.82
0.61
0.72
0.84
0.63
0.47
0.53
0.74
0.60
0.66
0.67
0.64
0.66
0.67
0.68
0.67
0.64
0.67
0.63
0.66
0.15*
0.03
0.06
0.17*
0.05
0.20*
0.11*
0.07
0.03
0
* Indicates significance at the p ¼ .05 threshold. ** Indicates significance at the p ¼ .01 threshold.
confirm this interaction (between the personality of the target and the
persuasion principle used) also occurs in the context of phishing emails.
In addition, the consistency with which extroversion predicted suscep­
tibility to phishing emails is notable.
Users’ susceptibility to the various Cialidini persuasion principles (or
combinations of principles), in an email phishing context, has been
subject to relatively limited investigation (Ferreira et al., 2015; Ferreira
and Teles, 2019; Parsons et al., 2019). We observed a strong scepticism
of emails using the authority & scarcity principle, and a trust of emails
using the liking principle, regardless of whether these principles were
used in a phishing or legitimate context. In the context of legitimate
emails, the combination of authority & scarcity evoked the greatest false
alarm rate, being perceived as a phishing attempt in greater than half of
cases. Knowing the likelihood that a user will identify a legitimate email
as a phishing attempt may be of interest to organizations, in that they
can attempt to avoid persuasion principles that arouse suspicion, such as
authority & scarcity, in outgoing emails.
The observed confidence data align well with other reported findings
from this study. For instance, participants who misidentified a legiti­
mate email using either the authority or authority & scarcity principles
as phishing were more confident in those responses than those who
answered correctly. Conversely, participants who misidentified a
phishing email using the authority principle as legitimate were less
confident in that response than those who answered correctly. This
pattern further shows a tendency to perceive emails using the authority
or scarcity principles as phishing attempts, whether that is the case or
not.
Similar but opposite patterns were observed when looking at emails
using the liking principle. Participants who misidentified a phishing
Table 8
Signal detection theory analysis.
Persuasion
Principle
d’
Miss
Rate
False
Alarm
Rate
Miss Rate –
False
Alarm Rate
Decision
Criterion
Status
Authority
Authority &
Commitment/
Consistency
Authority & Liking
Authority &
Scarcity
Commitment/
Consistency
Liking
Liking &
Commitment/
Consistency
Scarcity
Super (3þ)
1.23
.412
.178
.388
.377
.449
-.199
-.061
Liberal
Liberal
1.04
.888
.280
.161
.324
.541
-.043
-.380
Liberal
Liberal
.619
.369
.388
-.019
Liberal
.464
.509
.533
.333
.292
.469
.241
-.136
Conservative
Liberal
.989
.627
.255
.356
.371
.398
-.116
-.042
Liberal
Liberal
hypothesized), authority & commitment/consistency, authority &
liking, and commitment/consistency. There were seven instances where
personality was predictive of accuracy identifying emails utilizing spe­
cific persuasion principles, in both phishing and legitimate emails. An
interaction between the persuasion principle utilized and the target’s
personality characteristics is therefore broadly supported. Previous
research has demonstrated an interaction between the personality of the
target and the persuasion principle utilized in real-world social engi­
neering (Uebelacker and Quiel, 2014). The findings presented here
Fig. 7. Signal detection curve for the authority & scarcity persuasion principle.
Fig. 8. Signal detection curve for the liking persuasion principle.
8
P. Lawson et al.
Applied Ergonomics 86 (2020) 103084
email using the liking principle as legitimate were significantly more
confident in that response than those who answered correctly. Addi­
tionally, participants who incorrectly identified a legitimate email using
the liking persuasion principle as phishing were significantly less
confident than when the inverse occurred (when they misidentified a
phishing email using the liking principle as legitimate). Overall, par­
ticipants were more confident in these responses than they were accu­
rate, showing a tendency to label emails using the liking principle as
legitimate, even when that is not the case.
These differential response patterns to the various persuasion prin­
ciples are reflected in and corroborated by the placement of the decision
criterion in the signal detection theory analysis. For instance, partici­
pants demonstrated a high level of trust in emails utilizing the liking
principle, placing a highly conservative decision criterion, requiring
overwhelming evidence to label these emails as phishing attempts. On
the opposite end of the spectrum, emails utilizing the authority &
scarcity principle seemed to evoke a high level of distrust, resulting in a
highly liberal decision criterion, and a high tendency to label these as
phishing attempts (whether correctly or incorrectly).
It is interesting that authority is the principle most used by “phishermen”, despite this principle being one of the most likely (along with
scarcity) to arouse suspicion and be correctly identified as a phishing
attempt (Akbar, 2014; Williams et al., 2018). This may be explainable
simply by the goal of phishing, wherein the phisher must create an ur­
gency to reveal personal data. This study’s findings, however, reveals
that phishing emails utilizing only the liking persuasion principle are
least likely to be detected by the recipient. It may be beneficial to be
warier of such emails, given their potential to be well-disguised phishing
attempts. These findings may be especially valuable to the construction
of an email phishing susceptibility model. Such a model may be used to
better understand, predict, and potentially mitigate users’ susceptibility
to phishing emails. This is a major aim the authors plan to address in
future works.
Parsons et al., 2019), we have, after careful consideration, elected to
include our full findings and not omit results that could theoretically be
abused. It is the opinion of the authors that the benefits of such
knowledge outweigh the potential for misuse. For instance, it is not
possible to develop and subsequently utilize a phishing susceptibility
model to protect potential targets without first identifying the areas of
greatest susceptibility. Additionally, bad actors would likely find it
difficult to apply the presented research in harmful ways.
This decision is in part because only emails making use of the liking
persuasion principle and no other principles resulted in a conservative
decision criterion (i.e. participants had a tendency to trust them and
assume them to be legitimate emails). Given that these emails do not
demand an action be completed to avoid some consequence (or they
would be categorized as containing the authority principle), and do not
indicate there is a time frame for completion (or they would be cate­
gorized as containing the scarcity principle), it is likely that any ex­
change of sensitive information would have to occur in a subsequent
interaction, increasing requisite effort on the part of the phisher and
decreasing odds of success. This is to say that accidently believing a
phishing email using only the liking principle to be a legitimate email
does not necessarily imply that sensitive data is immediately at risk.
Perhaps it is for this reason that phishing emails are most likely to
contain the authority persuasion principle (Akbar, 2014; Williams et al.,
2018), given how directly and aggressively this principle seeks the
sensitive data. All things considered, we believe the present study and
those addressing similar phishing susceptibility models are absolutely
necessary to the creation of an informed system to identify and thwart
phishing attempts. We believe the potential for misuse is outweighed by
potential defensive benefits of such research.
5. Conclusions
High extroversion was confirmed to be highly predictive of suscep­
tibility to phishing emails. Furthermore, it was confirmed that there are
interaction effects between the personality of the victim and the
persuasion principle utilized in the context of phishing attacks. The most
effective persuasion principles to utilize in both phishing and legitimate
emails were also presented. The liking persuasion principle was
considered trustworthy in both phishing and legitimate emails.
Conversely, the combination of authority & scarcity persuasion princi­
ples was most likely to arouse suspicion in both phishing and legitimate
emails. These findings demonstrate clear, differential response patterns
when participants encountered emails utilizing the studied persuasion
principles.
4.1. Limitations
This study was affected by decreased ecological validity, stemming
from participants being explicitly instructed to identify whether emails
were legitimate or phishing attempts. This presumably heightened their
vigilance. This is evidenced by phishing detection accuracy being
significantly higher than legitimate detection accuracy; the participants
seem to have expected phishing emails, and therefore tended to err on the
side of false alarms (classifying a legitimate email as phishing) rather
than misses (classifying a phishing email as legitimate). It is likely that
phishing susceptibility might be higher in the “real world” given the
assumed decrease in vigilance. In effect, the priming in the current study
might represent a best-case scenario with regard to vulnerability.
Additionally, because the emails used as stimuli were real emails and
were not specially created for the purposes of this experiment, there was
limited control over the characteristics of these emails. The decision was
made to opt for ecological validity over perfect control. The phishing
and legitimate emails, for instance, were drawn from different sources:
phishing emails from a compendium of verified phishing emails, legit­
imate emails from the experimenters’ inboxes (as no equivalent com­
pendium existed for such emails). This means that it was not possible to
control for all potentially relevant variables, such as presence of spelling
or grammatical mistakes, inclusion of images, and general tone.
Declaration of competing interest
The authors declare that they have no known competing financial
interests or personal relationships that could have appeared to influence
the work reported in this paper.
Acknowledgements
This material is based upon work supported through the United
States National Security Agency under grant number 1318323.
Appendix A. Supplementary data
Supplementary data to this article can be found online at https://doi.
org/10.1016/j.apergo.2020.103084.
4.2. Ethicality
In this manuscript we present findings regarding susceptibility to a
variety of persuasion principles and combinations of principles. Because
high susceptibility for the recipient is synonymous with high efficacy
from the perspective of the phisher, the potential for abuse of this
research must be considered. As other researchers investigating phishing
susceptibility before us (Ferreira et al., 2015; Ferreira and Teles, 2019;
References
Akbar, N., 2014. Unpublished Master’s Thesis. Analysing Persuasion Principles in Phishing
Emails.
Benjamini, Y., Hochberg, Y., 1995. Controlling the false discovery rate: a practical and
powerful approach to multiple testing. J. Roy. Stat. Soc. B 289–300.
9
P. Lawson et al.
Applied Ergonomics 86 (2020) 103084
Sheng, S., Holbrook, M., Kumaraguru, P., Cranor, L.F., Downs, J., 2010. April). Who falls
for phish?: a demographic analysis of phishing susceptibility and effectiveness of
interventions. In: Proceedings of the SIGCHI Conference on Human Factors in
Computing Systems. ACM, pp. 373–382.
Sheng, S., Magnien, B., Kumaraguru, P., Acquisti, A., Cranor, L.F., Hong, J., Nunge, E.,
2007, July. Anti-phishing phil: the design and evaluation of a game that teaches
people not to fall for phish. In: Proceedings of the 3rd Symposium on Useable
Privacy and Security. ACM, pp. 88–99.
Sulistyawati, K., Wickens, C.D., Chui, Y.P., 2011. Prediction in situation awareness:
confidence bias and underlying cognitive abilities. Int. J. Aviat. Psychol. 21 (2),
153–174.
Tupes, E.C., Christal, R.E., 1961. Recurrent Personality Factors Based on Trait Ratings.
PsycEXTRA Dataset.
Uebelacker, S., Quiel, S., 2014, July. The social engineering personality framework. In:
2014 Workshop on Socio-Technical Aspects in Security and Trust. IEEE, pp. 24–30.
Wang, J., Li, Y., Rao, H.R., 2016. Overconfidence in phishing email detection. J. Assoc.
Inf. Syst. Online 17 (11), 759–783.
Welk, A.K., Hong, K.W., Zielinska, O.A., Tembe, R., Murphy-Hill, E., Mayhorn, C.B.,
2015. Will the “phisher-men” reel you in?: assessing individual differences in a
phishing detection task. Int. J. Cyber Behav. Psychol. Learn. (IJCBPL) 5 (4), 1–17.
Williams, E.J., Hinds, J., Joinson, A.N., 2018. Exploring susceptibility to phishing in the
workplace. Int. J. Hum. Comput. Stud. 120, 1–13.
Workman, M., 2008. Wisecrackers: a theory-grounded investigation of phishing and
pretext social engineering threats to information security. J. Am. Soc. Inf. Sci.
Technol. 59 (4), 662–674.
Zielinska, O.A., Welk, A.K., Mayhorn, C.B., Murphy-Hill, E., 2016, September.
A temporal analysis of persuasion principles in phishing emails. In: Proceedings of
the Human Factors and Ergonomics Society Annual Meeting, vol 60. SAGE
Publications, pp. 765–769. No. 1.
Canfield, C.I., Fischhoff, B., Davis, A., 2016. Quantifying phishing susceptibility for
detection and behavior decisions. Hum. Factors 58 (8), 1158–1172.
Cialdini, R.B., 1987. Influence, vol 3. A. Michel.
Costa, P.T., McCrae, R.R., 1992a. Normal personality assessment in clinical practice: the
NEO personality inventory. Psychol. Assess. 4 (1), 5–13.
Costa, P.T., McCrae, R.R., 1992b. Four ways five factors are basic. Pers. Indiv. Differ. 13
(6), 653–665.
Costa, P.T., McCrae, R.R., 1992c. Neo PI-R Professional Manual.
Erdheim, J., Wang, M., Zickar, M.J., 2006. Linking the Big Five personality constructs to
organizational commitment. Pers. Indiv. Differ. 41 (5), 959–970.
Ferreira, A., Coventry, L., Lenzini, G., 2015, August. Principles of persuasion in social
engineering and their use in phishing. In: International Conference on Human
Aspects of Information Security, Privacy, and Trust. Springer International
Publishing, pp. 36–47.
Ferreira, A., Teles, T., 2019. Persuasion: how phishing emails can influence users and
bypass security. Int. J. Hum. Comput. Stud. 125, 19–31.
Green, D.W., Swets, J.A., 1966. Signal Detection Theory and Psychophysics. John Wiley
& Sons, New York.
Jagatic, T.N., Johnson, N.A., Jakobsson, M., Menczer, F., 2007. Social phishing.
Commun. ACM 50 (10), 94–100.
Kumaraguru, P., Cranshaw, J., Acquisti, A., Cranor, L., Hong, J., Blair, M.A., Pham, T.,
2009. July). School of phish: a real-world evaluation of anti-phishing training. In:
Proceedings of the 5th Symposium on Useable Privacy and Security. ACM, p. 3.
Mayhorn, C.B., Nyeste, P.G., 2012. Training users to counteract phishing. Work 41
(Suppl. 1), 3549–3552.
Parrish Jr., J.L., Bailey, J.L., Courtney, J.F., 2009. A Personality Based Model for
Determining Susceptibility to Phishing Attacks. Little Rock: University of Arkansas.
Parsons, K., Butavicius, M., Delfabbro, P., Little, M., 2019. Predicting susceptibility to
social influence in phishing emails. Int. J. Hum. Comput. Stud. 128, 17–26.
10