Privacy & Stylometry: Practical Attacks Against Authorship
Recognition Techniques
Michael Brennan and Rachel Greenstadt
{mb553,greenie}@cs.drexel.edu
Overview
Introduction to Stylometry/Authorship Recognition
The Threat to Privacy & Anonymity



Experimental Results
Attacking Authorship Recognition




Study Setup
Methodology
Results
Future Work

2
What is Authorship Recognition?


The basic question: “who wrote this document?”
Stylometry: The study of attributing authorship to
documents based only on the linguistic style they exhibit.



“Linguistic Style” Features: sentence length, word choices,
syntactic structure, etc.
Handwriting, content-based features, and contextual features
are not considered.
In this presentation, stylometry and authorship
recognition are used interchangeably.
Stylometry: A Brief History


The classic stylometry problem: The Federalist Papers.
 85 anonymous papers to persuade ratification of the
Constitution. 12 of these have disputed authorship.
 Stylometry has been used to show Madison authored
the disputed documents.
 Used as a data set for countless stylometry studies.
Modern Stylometry Based in Machine Learning
 SVMs, Genetic Algorithms, Neural Networks, Bayesian
Classifiers… used extensively.
Why is Stylometry Important?


Great, you can figure out who wrote a 200 year old
document, so what?
From the Institute for Linguistic Evidence:


“In some criminal, civil, and security matters, language can be
evidence… When you are faced with a suspicious document,
whether you need to know who wrote it, or if it is a real
threat or real suicide note, or if it is too close for comfort to
some other document, you need reliable, validated methods.”
Plagiarism, Forensics, Anonymity…
Stylometry: the Threat to Privacy
Good techniques for location privacy (Tor, Mixes, etc).


But it may be insufficient!
Stylometry can identify authors based on their writing.




6
Can anonymous authors defend against this?
~6500 words to leak identity – Rao, Rohatgi. 2000.
“The Multidisciplinary Requirement for Privacy” – Carlisle
Adams. 2006.
Supervised Stylometry
Given a set of documents of known authorship, classify a
document of unknown authorship.


Classifier trained on undisputed text.
Scenario: Alice the Anonymous Blogger vs. Bob the
Abusive Employer.


Alice blogs about abuses at Bob’s company.


Bob obtains 5000-10000 words of each employee’s writing.

7
Blog posted anonymously (Tor, pseudonym, etc).
Bob uses stylometry to identify Alice as the blogger.
Unsupervised Stylometry
Given a set of documents of unknown authorship, cluster
them into author groups.



No pre-existing author information.
“Similarity Detection”
Scenario: Anonymous Forum vs. Oppressive Government.


Participants organize protests.



The government applies unsupervised stylometric techniques.


8
Posts are completely unlabeled (no pseudonyms)
Unknown organizational structure, number of authors, etc.
Number of authors may be discovered, author profiles created.
Results fed into supervised stylometry system to identify individuals.
The Threat in Numbers (Unsupervised)
9 Anonymous authors, raw forum data:



Only 35% Accuracy.
Posts 25-200 words long
5 authors, same data set, modified for longer posts:



88.2% Accuracy
Passages 500-750 words long
Other Research on 50 Unique Authors (WritePrints):


9
Unsupervised “Pairing”: 95%
The Threat in Numbers (Supervised)

Supervised Classification of 2-5 Authors: 79-99% Accuracy
(IAAI ‘09).



Random sets taken from 15 subject pool.
3 Different Methods Used
Other Research on 50 Unique Authors (WritePrints):

10
Supervised: 90%
Protecting Privacy: Attacking Stylometry

Problem


Evaluation


Can stylometry be attacked? How so? How Easily?
In depth study on attacking multiple methods of Stylometry.
Conclusion


11
Stylometry is very vulnerable to attack by inexperienced
human adversaries.
Attacks can be used to protect privacy.
Related Work

Adversarial stylometry is not a well-researched part of
the field.



Somers & Tweedie, 2003: Authorship Attribution & Pastiche.
Mixed Results.
Kacmarcik & Gamon, 2006: Obfuscating Document Stylometry
to Preserve Anonymity. Encouraging results.
Our goal: Assess the general vulnerability of authorship
recognition systems when under attack by human
adversaries.
We’re Under Attack!

Obfuscation Attack


An author attempts to write a document in such a way that
their personal writing style will not be recognized.
Imitation Attack

An author attempts to write a document such that the writing
style will be recognized as that of another specific author.
Study Setup & Format


3 representative methods of stylometry.
15 Individual Authors. Participation had three parts:


Submit 5000 words of pre-existing writing from a formal
source.
Write a new 500 word passage as an obfuscation attack.


Write a new 500 word passage as an imitation attack.


Task: Describe your neighborhood.
Task: Imitate Cormac McCarthy, describe your day.
Authors had no formal training or knowledge in
linguistics or stylometry.
Imitating Cormac McCarthy
“On the far side of the river valley the
road passed through a stark black burn.
Charred and limbless trunks of trees
stretching away on every side. Ash
moving over the road and the sagging
hands of blind wire strung from the
blackened lightpoles whining thinly in
the wind.”
Imitation Attack Examples



“Light sliced through the blinds, and construction began in
the adjacent apartment. The harsh cacophony crashed
through the wall.”
“Hot water in the mug. Brush in the mug. The blade read
‘Wilkinson Sword’ on the layered wax paper packaging.”
“He fills the coffee pot with water, after cleaning out the
putrid remains of yesterday's brew. The beans are in the
freezer, he remembers.”
Methodology & Evaluation



Validation Experiment:Verify accuracy claims of all 3
methods.
Attack Experiment: Determine accuracy when attacked
using the same experimental conditions.
Experimental Conditions:

Each test was applied to four random sets of 2, 3, 4, and 5
distinct authors.


Same sets used in each method.
Cross-validation used to measure accuracy (repeated sub
sampling and leave-one-out).
Machine Learning: Quick Introduction



Machine Learning is AI
 Learn & recognize patterns automatically, make
decisions.
Most common incarnation of ML: Classifiers
 Spam Filter: Bayesian Classifier
 Given a message, what is the probability that it is spam
or not spam given the extracted features.
Important to pick good features.
A Closer Look at Linguistic Features



Basic Measurements:
 Average syllable/word/sentence count, letter
distribution, punctuation.
Lexical Density
 Unique_Words / Total_Words
Gunning-Fog Readability Index:
 0.4 * ( Average_Sentence_Length +
100 * Complex_Word_Ratio )
 Result: years of formal education required to read the
text.
Stylometry Methods

Method 1: Signature Stylometric System.

Three features: word length, letter usage, punctuation usage.
95% base accuracy.
Stylometry Methods

Method 2: Neural Network Classifier


Based on Singh & Holmes, 1996.
9 linguistic features.



Lexical density, unique word count, character counts with and without
spaces, two readability indexes, sentence count, average sentence
length, and average syllables per word.
78.5% base accuracy.
NN: simple switches,
complex overall system.
Input
Hidden
Output
Stylometry Methods

Method 3: Synonym-Based Classifier.


Based on Clark & Hannon, 2007.
Examines word choice when presented with all possible
synonyms. 88-99% accuracy.
(Diagram from “A Classifier System for Author Recognition
Using Synonym-Based Features” by Clark & Hannon)
Accuracy on Training Set
All three methods were highly accurate in recognizing the correct author of the training set.
Obfuscation Attack Detection
Obfuscation attacks brought the accuracy of each system to that of random chance or worse.
Imitation Attack Detection
Imitation attacks all but completely circumvented all three authorship recognition methods.
Imitation Attack Success
The imitation attacks were very successful in targeting the intended victim of the attack.
Questioning the Results





Why are some authors more susceptible to attack?
Why did certain methods perform better than others?
Would these results hold up in a stricter domain?
What if the participants were skilled in linguistics or
stylometry?
What features and methods could offer some resistance
to these attacks?
Using Attacks to Improve Privacy

What you can do:




Important: These these methods failed, others might not.
Best: Try to imitate someone else.
Write less.
What is needed

28
A tool that will assist in anonymizing writing styles.
Building a Tool

What a style-modifying tool needs:




A variety of methods & features.
A large corpus of existing authors.
AI-Assisted (Human oversight to preserve content)
Non-Stylometry Features:


Contextual Clue Identifiers (Time, Content)
Building a Corpus



IRB Approval
Multiple Languages
Your help!
Future Work





Testing more stylometric methods.
Imitation attacks against other authors.
Effectiveness in unique domains (aggregated short
messages)
Extracting additional information (demographic)
Better demonstration of effective unsupervised
stylometry.
Conclusion

Not the end of stylometry in sensitive areas


Stylometry is useful, but can also present a threat to
privacy.


New methods should test for adversarial threats.
Attacking stylometry to preserve privacy has high potential.
Potential for arms race:

Developing attack-resistant methods of stylometry vs. creating
new attacks to preserve privacy.
Questions?

Contact:




Mike Brennan: mb553@drexel.edu
Rachel Greenstadt: greenie@cs.drexel.edu
www.cs.drexel.edu/~{mb553,greenie}
Interested in learning more?



“Can Pseudonymity Really Guarantee Privacy?” – Josyula Rao,
Pankaj Rohatgi. 2000.
“The Multidisciplinary Requirement for Privacy” – Carlisle
Adams. 2006.
“Writeprints: A stylometric approach to identity-level
identification and similarity detection in cyberspace” – Ahmed
Abbasi, Hsinchun Chen. 2008.
FAQ




“When will your data set be released?”
“I can think of features for a stylometry system that I bet
these authors did not consider”
“I know a person in a certain Three Letter Organization
that has subtle ways of detecting authorship that would
surely not be broken by these attacks.”
“What about function words?”
Message Board Posts vs. Original Data
Study Setup: Detailed


3 representative methods of stylometry.
15 Individual Authors. Participation had three parts:

Submit 5000 words of pre-existing writing from a formal source.


Write a new 500 word passage as an obfuscation attack.


Task: Describe your neighborhood.
Write a new 500 word passage as an imitation attack.


Formal means school essays, professional reports, etc. No slang,
abbreviations, casual conversation.
Task: Imitate Cormac McCarthy using a passage from The Road, write a
third person narrative about your day starting from when you wake up.
Authors had no formal training or knowledge in linguistics or
stylometry.
Discussion

Training Set Content and Size

The amount of training text for each author not exceptionally large,
but the number of authors is significantly larger than most studies.




Could be beneficial to study the effects of adversarial attacks in
stricter domains.
Participant Skill Level


Allowed us to see interesting patterns, such as authors who were better
at creating attacks.
Certain authors particularly susceptible to obfuscation attacks. Do they
have a “generic” writing style?
The lack of linguistic expertise of the participants strengthens our
conclusions. It would be reasonable to expect authors with some
level of expertise to do a better job at attacking these methods.
Examining what features and methods might offer resistance to
these attacks is a viable avenue of research.