Deceptive Speech
Frank Enos • April 19, 2006
Defining Deception

Deliberate choice to mislead a target
without prior notification (Ekman‘’01)

Often to gain some advantage

Excludes:
 Self-deception
 Theater, etc.
 Falsehoods due to ignorance/error
 Pathological behaviors
Why study deception?

Law enforcement / Jurisprudence

Intelligence / Military / Security

Business

Politics

Mental health practitioners

Social situations
 Is it ever good to lie?
Why study deception?

What makes speech “believable”?

Recognizing deception means recognizing
intention.

How do people spot a liar?

How does this relate to other subjective
phenomena in speech? E.g. emotion,
charisma
Problems in studying deception?

Most people are terrible at detecting deception
— ~50% accuracy
(Ekman & O’sullivan 1991, Aamodt 2006, etc.)

People use subjective judgments —
emotion, etc.

Recognizing emotion is hard
People Are Terrible At This
Group
#Studies
#Subjects
Accuracy %
Criminals
1
52
65.40
Secret service
1
34
64.12
Psychologists
4
508
61.56
Judges
2
194
59.01
Cops
8
511
55.16
Federal officers
4
341
54.54
122
8,876
54.20
Detectives
5
341
51.16
Parole officers
1
32
40.42
Students
Problems in studying deception?

Hard to get good data
 Real world (example)
 Laboratory

Ethical issues
 Privacy
 Subject rights
 Claims of success

But also ethical imperatives:
 Need for reliable methods
 Debunking faulty methods
 False confessions
20th Century Lie Detection

Polygraph
 http://antipolygraph.org
 The Polygraph and Lie Detection (N.A.P. 2003)

Voice Stress Analysis
 Microtremors 8-12Hz
 Universal Lie response
 http://www.love-detector.com/
 http://news-info.wustl.edu/news/page/normal/669.html

Reid
 Behavioral Analysis Interview
 Interrogation
An Example…
Frank Tells Some Lies
Frank Tells Some Lies
Maria: I’m buying tickets to Händel’s Messiah for me
and my friends — would you like to join us?
Frank: When is it?
Maria: December 19th.
Frank: Uh… the 19th…
Maria: My two friends from school are coming, and
Robin…
Frank: I’d love to!
How to Lie (Ekman‘’01)

Concealment

Falsification

Misdirecting

Telling the truth falsely

Half-concealment

Incorrect inference dodge.
Frank Tells Some Lies
Maria: I’m buying tickets to Handel’s Messiah for me
and my friends — would you like to join us?
Frank: When is it?
Maria: December 19th.
Frank: Uh… the 19th…
Maria: My two friends from school
are coming, and Robin…
Frank: I’d love to!
• Concealment
• Falsification
• Misdirecting
• Telling the truth falsely
• Half-concealment
• Incorrect inference dodge.
Reasons To Lie (Frank‘’92 )

Self-preservation

Self-presentation

*Gain

Altruistic (social) lies
How Not To Lie (Ekman‘’01)

Leakage
 Part of the truth comes out
 Liar shows inconsistent emotion
 Liar says something inconsistent with the lie

Deception clues
 Indications that the speaker is deceiving
 Again, can be emotion
 Inconsistent story
How Not To Lie (Ekman‘’01)

Bad lines





Lying well is hard
Fabrication means keeping story straight
Concealment means remembering what is omitted
All this creates cognitive load  harder to hide emotion
Detection apprehension (fear)





Target is hard to fool
Target is suspicious
Stakes are high
Serious rewards and/or punishments are at stake
Punishment for being caught is great
How Not To Lie (Ekman‘’01)

Deception guilt
 Stakes for the target are high
 Deceit is unauthorized
 Liar is not practiced at lying
 Liar and target are acquainted
 Target can’t be faulted as mean or gullible
 Deception is unexpected by target

Duping delight
 Target poses particular challenge
 Lie is a particular challenge
 Others can appreciate liar’s performance
Features of Deception

Cognitive
 Coherence, fluency

Interpersonal
 Discourse features: DA, turn-taking, etc.

Emotion
Describing Emotion

Primary emotions
 Acceptance, anger, anticipation, disgust, joy,
fear, sadness, surprise

One approach:
continuous dim. model (Cowie/Lang)

Activation – evaluation space

Add control/agency

Primary E’s differ on at least 2 dimensions of this
scale (Pereira)
Problems With
Emotion and Deception

Relevant emotions may not differ much on
these scales

Othello error
 People are afraid of the police
 People are angry when wrongly accused
 People think pizza is funny

Brokow hazard
 Failure to account for individual differences
Bulk of extant deception research…

Not focused on verifying 20th century
techniques

Done by psychologists

Considers primarily facial and physical cues

“Speech is hard”

Little focus on automatic detection of
deception
Modeling Deception in Speech

Lexical

Prosodic/Acoustic

Discourse
Deception in Speech (Depaulo ’03)

Positive Correlates
 Interrupted/repeated words
 References to “external” events
 Verbal/vocal uncertainty
 Vocal tension
 F0
Deception in Speech (Depaulo ’03)

Negative Correlates
 Subject stays on topic
 Admitted uncertainties
 Verbal/vocal immediacy
 Admitted lack of memory
 Spontaneous corrections
Problems, revisited

Differences due to:
 Gender
 Social Status
 Language
 Culture
 Personality
Columbia/SRI/Colorado Corpus

With Julia Hirschberg, Stefan Benus, and
colleagues from SRI/ICSI and U. C. Boulder

Goals
 Examine feasibility of automatic deception
detection using speech
 Discover or verify acoustic/prosodic, lexical,
and discourse correlates of deception
 Model a “non-guilt” scenario
 Create a “clean” corpus
Columbia/SRI/Colorado Corpus

Inflated-performance scenario

Motivation: financial gain
and self-presentation

32 Subjects: 16 women, 16 men

Native speakers of Standard American English

Subjects told study seeks to identify people who
match profile based on “25 Top Entrepreneurs”
Columbia/SRI/Colorado Corpus

Subjects take test in six categories:
 Interactive, music, survival, food,
NYC geography, civics

Questions manipulated 
 2 too high; 2 too low; 2 match

Subjects told study also seeks people who can
convince interviewer they match profile
 Self-presentation + reward

Subjects undergo recorded interview in booth
 Indicate veracity of factual content of each utterance using
pedals
CSC Corpus: Data

15.2 hrs. of interviews; 7 hrs subject speech

Lexically transcribed & automatically aligned 
lexical/discourse features

Lie conditions: Global Lie / Local Lie

Segmentations (LT/LL):
slash units (5709/3782), phrases (11,612/7108),
turns (2230/1573)

Acoustic features (± recognizer output)
Columbia University– SRI/ICSI – University of Colorado
Deception Corpus: An Example Segment
SEGMENT TYPE
Breath Group
LABEL
LIE
Obtained
from subject
pedal presses.
um i was visiting a friend in venezuela and we went camping
ACOUSTIC FEATURES
max_corrected_pitch
mean_corrected_pitch
Produced using
ASR output
and other
acoustic analyses
pitch_change_1st_word
pitch_change_last_word
normalized_mean_energy
unintelligible_words
5.7
5.3
-6.7
-11.5
0.2
0.0
LEXICAL FEATURES
has_filled_pause
positive_emotion_word
uses_past_tense
PREDICTION
negative_emotion_word
contains_pronoun_i
verbs_in_gerund
YES
YES
NO
LIE
NO
YES
YES
Produced
automatically
using lexical
transcription.
CSC Corpus: Results

Classification
(Ripper rule induction, randomized 5-fold cv)
 Slash Units / Local Lies — Baseline 60.2%
 Lexical & acoustic: 62.8 %; + subject dependent: 66.4%
 Phrases / Local Lies — Baseline 59.9%
 Lexical & acoustic 61.1%; + subject dependent: 67.1%

Other findings




Positive emotion words deception (LIWC)
Pleasantness  deception (DAL)
Filled pauses  truth
Some pitch correlation — varies with subject
Example JRIP rules:
(cueLieToCueTruths >= 2) and (TOPIC = topic_newyork) and
(numSUwithFPtoNumSU <= 0) and
(wu_ENERGY_NO_UV_STY_MAX__EG_ZNORMENERGY_NO_UV_STY_MIN__EG_ZNORM-D <= 5.846)
=> PEDAL=L (231.0/61.0)
(cueLieToCueTruths >= 2) and (numSUwithFPtoNumSU <= 1) and
(wu_ENERGY_NO_UV_STY_MAX__EG_ZNORMENERGY_NO_UV_STY_MIN__EG_ZNORM-D <= 5.68314) and
(wu_ENERGY_NO_UV_RAW_MAX-ENERGY_NO_UV_RAW_MIN-D >= 8.41605)
and (wu_F0_SLOPES_NOHD__LAST >= -2.004)
=> PEDAL=L (284.0/117.0)
(cueLieToCueTruths >= 2) and (wu_F0_RAW_MAX >= 5.706379) and
(wu_DUR_PHONE_SPNN_AV <= 1.0661)
=> PEDAL=L (262.0/115.0)
CSC Corpus: A Perception Study

With Julia Hirschberg, Stefan Benus, Robin Cautin
and colleagues from SRI/ICSI

32 Judges

Each judge rated 2 interviews

Judge Labels:
 Local Lie using Praat
 Global Lie on paper

Takes pre- and post-test questionnaires

Personality Inventory

Judge receives ‘training’ on one subject.
By Judge
58.2% Acc.
By Interviewee
Personality Measure: NEO-FFI

Costa & McCrae (1992) Five-factor model
 Openness to Experience
 Conscientiousness
 Extraversion
 Agreeability
 Neuroticism

Widely used in psychology literature
Neuroticism, Openness & Agreeableness
correlate with judge performance
WRT Global lies.
These factors
also provide
strongly
predictive
models for
accuracy at
global lies.
Other Perception Findings

No effect for training

Judges’ post-test confidence did not correlate
with pre-test confidence

Judges who claimed experience had
significantly higher pre-test confidence
 But not higher accuracy!

Many subjects used disfluencies as cues to D.
 In this corpus, disfluencies correlate with TRUTH!
(Benus et al. ‘06)
Our Future Work

Individual differences
 Wizards of deception

Predicting Global Lies
 Local lies as ‘hotspots’

New paradigm
 Shorter
 Addition of personality test for speakers
 Addition of cognitive load