National Security and Social Media Monitoring
Extracting the Meaning Of Terse Information in a Visualisation of Emotion
13th August, EISIC 2013, Uppsala, Sweden
Dr. Martin Sykora, Prof. Tom Jackson, Dr. Ann O’Brien and
Dr. Suzanne Elayan
Contents

Introduction

5 Systems Overview (common featurs.)

EMOTIVE System & Future Work

Conclusions
Introduction
Systems Overview
EMOTIVE
Conclusions
Egyptian activist; “We use Facebook to schedule our protests, Twitter
to coordinate and YouTube to tell the world.” (Meier 2011)
Social Media – Polling public opinion: O’Connor et al. 2010,
Tumasjan et al. 2010, Cheong et al. 2011, Lansdall-Welfare et al. 2012.
Social Media is first to break-the-news!
- 2008 Mumbai attacks, where individuals on location broke the news via Twitter.
- July 2009 Jakarta bombings, where Twitter broke the news.
- Even earthquakes, ranging from seismic intensity scale 3 or more, were reported
quicker by Twitter users as opposed to the relevant Japanese agency.
Commercial Interest: Attensity, Crimson Hexagon, Sysomos,
Brandwatch, Vocus, Socialradar, Radian 6, ...
Crisis mapping community: http://crisismappers.net
…in this talk, briefly look at 5 systems: CrisisTracker,
Crisees, SensePlace2, Swiftriver (Ushahidi), Twitcident.
Introduction
Systems Overview
EMOTIVE
Conclusions
Prior literature highlights: (Cheong and Lee 2011, Glass and Colbaugh 2012)
• importance of gauging public response to terrorism
events from social media
• ..and specifically highlights importance of automatic
sentiment detection in Tweets
Automated monitoring systems (tools & techniques) are
necessary to deal with the big data in social streams.
Introduction
Systems Overview
EMOTIVE
Conclusions
Unfortunately (“fortunately”) there is a very wide range of different content
and styles of messages communicated on Twitter, which requires
selective interpretation of the messages in a National Security setting.
It is apparent (Rogstadius et al. 2011, Johansson et al. 2012, Cheong and Lee 2011) that,
..to facilitate monitoring of a certain event or entity of interest,
1-efficient extraction of messages, 2-geo-location, 3-emotion
evaluation, 4-clustering and organization of the tweet messages,
and an 5-intuitive user-interface are necessary.
Hence to facilitate effective national security monitoring tasks:
1.
2.
3.
4.
5.
Keyword / Keyphrase monitoring, first event detection, filtering and extraction
Accurate geo-location detection
Emotion detection and evaluation
Tone of tweet message detection, further semantic enrichment and organisation
User-interface visualisation
Introduction
1.
Systems Overview
EMOTIVE
Conclusions
Topic relevant messages retrieved & spam filtered out:
a) Manual input; keywords / phrases / #hashtags
b) keywords based on automated named entity recognition of regularly re-checked
trending twitter topics (Cheong & Lee 2011)
c) Using first story detection (Locality-Sensitive Hashing is popular) (Petrovic et al. 2011)
2, 3 and 4 further substantially enriches retrieved text messages by providing
context through:
2.
extracting location details
3.
extracting communicated emotions / sentiment
4.
extracting various features from the tone of the messages / sem. enrichment
…these 3 steps essentially steps in automated semantic enrichment [22]
5. real-time visualisation and a faceted user-interface to explore the enriched
sparse text message data
Introduction
Systems Overview
EMOTIVE
Conclusions
Introduction
Systems Overview
EMOTIVE
Conclusions
Introduction

Systems Overview
EMOTIVE
Conclusions
Emotion extraction, prior work:
 Notions of affect and sentiment have been rather simplified in
current state-of-the-art, often confined to their assumed overall
polarity (i.e. positive / negative), Thelwall (2012)

Another problem with polarity-centric sentiment classifiers is that
they generally encompass a vague notion of polarity that bundles
together emotion, states and opinion

There is no common agreement about which features are the most
relevant in the definition of an emotion and which are the relevant
emotions and their names, Grassi (2009)

1-machine learning; 2-lexicon / linguistic analysis; & 3-polarity estimation from term
co-occurrence (Thelwall et al. 2012)

Comparison:
de Choudhury and Counts (2012) & Thelwall et al. (2012)
Introduction

Systems Overview
EMOTIVE
Conclusions
EMOTIVE emotion detection discovers finegrained explicit emotions in sparse text messages:



Anger, Disgust, Fear, Happiness, Sadness, Surprise
(Ekman’s 6 basic emotions) + Shame, and Confusion.
Shame – common on Twitter
Confusion – useful for situational awareness, Oh et al. (2011)
Introduction
Systems Overview
EMOTIVE
Conclusions

The ontology contains over 300 emotional terms, with many
intensifier, conjunction, negation and interjection words and phrases.
It also contains information on the perceived strength of emotions,
and some linguistic analysis related information.

Example Emotion Terms from the Ontology:
Anger
Confusion
Disgust
Fear
Happiness
Sadness
Shame
Surprise
(e.g. enraged, infuriated, peeved, in a tizzy…)
(e.g. chaotic, distracted, perplexed, confuzzled…)
(e.g. appalling, beastly, bullshit, scuzzy…)
(e.g. cold feet, goose bumpy, petrified, scary…)
(e.g. blissful, chuffed, delighted, in high spirits…)
(e.g. depressed, devastating, duff, grief stricken…)
(e.g. abashment, degrading, hang head in shame, scandalous…)
(e.g. astonished, disbelief, gobsmacked, off guard…)
Introduction
Systems Overview
EMOTIVE
Conclusions
Introduction

Systems Overview
EMOTIVE
Conclusions
Study of language performed by an English language
and literature PhD level research associate, with training
in linguistics and discourse analysis, during a three
month time-period.




600MB of cleaned Tweets on 63 different UK-specific topics /
search-terms datasets
Focused on identifying commonly used explicit expressions of
emotion
OOV (Out of Vocabulary) terms, Wordnet synset synonym lists of
emotional expressions, Dictionary.com, Thesaurus.com, the
Oxford
English
online
dictionary, Urbandictionary.com,
Internetslang.com…
Emotional terms and activation levels identified and used in work
by Choudhury et al. (2012) and lexicon lists of intensifiers,
negators & words of basic sentiment used in SentiStrength-2,
Thelwall et al. (2012) were also reviewed.
Introduction

Systems Overview
EMOTIVE
Conclusions
On an initial golden-dataset (annotated by 2 human annotators) of emotive tweets the
technique achieved excellent results, F-measure = .962:
Recall, precision and f-measure, were
computed using an equivalent
approach as used in CoNNL-2003
shared task on NER (Tjong et al. 2003).

This is an extremely high f-measure illustrating the successful nature of the ontology.

To compare our high f-measure to another approach, fine-grained emotion detection
from Choudhury et al. (2012) achieved; .744 / .668 (f-measure), .830 / .658
(precision) and .674 / .680 (recall); direct matching / stemmed matching, respectively.

EMOTIVE’s emotion strength scoring approach was evaluated against SentiStrength-2
(Thelwall et al. 2012): a consistent and statistically significant correlation was found; which
indicates that we are measuring in line with a sentiment scoring state-of-the-art system.
Woolwich Soldier Killing (Lee Rigby) – #woolwich, Anjem Choudary
The brutal murder sparked a storm of emotional reactions of Sadness, Disgust
and Surprise. At the same time the controversial cleric Anjem Choudary was
most often mentioned with extreme emotions of Anger and Disgust.
Example reactions to Anjem Choudary
.I'm quite angry that Anjem Choudary is on Newsnight tonight - I can only imagine
how furious Muslims he falsely claims to speak for must be [anger]
.And I'm angry that Anjem Choudary is aloud to preach hate in our towns and city's
It's the government we should be angry with not a religion [anger]
.Anjem Choudary, gfy. Ruining the 'Choudhary' name for all of us, you complete
bastard, it's sickening #woolwich [disgust]
.@EDLTrobinson so sad, and so wrong that ANJEM CHOUDARY can get air time
saying muslims around the world will call them heroes what a twat. [sadness]
Woolwich Soldier Killing (Lee Rigby) – #woolwich
.Enjoyed last night's @HyderiCentre event in response to #Woolwich with
@cllrjudybest, @jon_bartley, @ihrc, Syed Ammar & Sheikh Panju.. [happiness]
.Great @LabourList article from @jonewilson on the town I'm proud to live in. We
love
#Woolwich
[happiness]
.My heart
goes tohttp://t.co/W0MXkxIqvm
the soldiers family, friends,
the people of #woolwich & all those
effected,
soof
pretty
much soldier
everyone.
news.Rigby:
[sadness]
.The family
murdered
payTerribly
tribute. sad
Rebecca
"I love Lee, I always will
and
I'm proud
to be Military
his wife."
#woolwich
.#Woolwich
Attack:
New
Shocking
Video [happiness]
of Terrorists
Charging
Police
Getting
.@SkyNewsBreak:
commanders
tell
soldiers told
not to at
wear
theirCar,
uniforms
.@steveplrose: "Free speech in Britain is threatened by the influence of Muslims in
Shot:
http://t.co/icVNVud5ci
via @youtube
#EDL
[surprise]
in
public
until further notice #Woolwich"
- Sad
times
:( [sadness]
the media" YouGov question. Wow. #woolwich http://t.co/NqFjpDqJpy. [fear]
.Utterly
seeasome
up claiming
#Woolwich
.The
factastonished
the soldiertowas
fathervideos
upsetspopping
me further.
Maybe itthe
shouldn't
but itattack
does
.Following the #Woolwich incident, people in #Britain are anxious. Reports of a man
was a hoax[sadness]
with the media and government colluding! [surprise]
#woolwich
with an axe at #LondonBridge is making people nervous. [fear]
.#woolwich absolutely disgusting scene yesterday. Jst so annoying [disgust]
.#Woolwich - so awful. Strength to the victim's families. [disgust]
.Can't believe the news about yesterday's #woolwich attack. Disgusting. Some
people are so sick!! [disgust]
.RT @skymartinbrunt: #woolwich She was arrested on Wednesday after apparently
asking police for protection when her malicious tweet prompted angry backlash.
[anger]
Dyka Ayan Hassan from Harrow, 21, arrested for a malicious tweet
Introduction
Systems Overview
Future Work
Conclusions
Funding – ReDites (Real Time, Detection, Tracking, Monitoring and Interpretation of Events in Social Media)
1.
Event detection & tracking
2.
Location profiling
3.
EMOTIVE emotions extraction
4.
Author profiling (tone of messages; first hand, bots, astroturfing, social network analysis)
5.
Event exploration and event summarisation / notification interface
Introduction
System Overview
Interpretation
Conclusions

Powerful (fine-grained) Emotion extraction from Tweets
is mostly missing in these types of Systems, although
prior work found it to be of significant importance!

Social media monitoring system – EMOTIVE

Excellent F-measure achieved & evaluated against two other systems

Aiding analysts to interpret live events
Learning and predicting from previous datasets


ReDites to tackle the highlighted System Elements –
Deliver a Demonstrator for National Security Monitoring
Thanks
References
Cheong M. and Lee V. C. S., 2011. A microblogging-based approach to terrorism informatics: Exploration and chronicling civilian sentiment and
response to terrorism events via Twitter, Journal of Information Systems Frontiers – Springer 13, pp. 45-59
Choudhury M. and Counts S., 2012. The Nature of Emotional Expression in Social Media: Measurement, Inference and Utility, Technical
Report: Microsoft.
Drummond T., 2004. Vocabulary of Emotions [Online], North Seattle Community College, [last viewed 9.1.2012]. Available from
http://www.sba.pdx.edu/faculty/mblake/448/FeelingsList.pdf
Ekman P., 1994. All emotions are basic. The nature of emotion: Fundamental questions 15-19.
Gimpel K., Schneider N., O'Connor B., Das D., Mills D., Eisenstein J., Heilman M., Yogatama D., Flanigan J. and Smith N., 2010. Part-ofspeech tagging for twitter: Annotation, features, and experiments, Technical Report.
Glass K. and Colbaugh R., 2012. Estimating the sentiment of social media content for security informatics applications, Security Informatics 1,
pp. 1-16
Grassi M., 2009. Developing HEO human emotions ontology, Biometric ID Management and Multimodal Communication, Springer Berlin
Heidelberg, pp. 244-251
Izard C. E., 2009. Emotion theory and research: Highlights, unanswered questions, and emerging issues. Annual Review of Psychology 60, 125.
O'Connor B., Krieger M. and Ahn D., 2010. TweetMotif: Exploratory Search and Topic Summarization for Twitter, Proceedings of the
International AAAI Conference on Weblogs and Social Media, Washington DC (USA)
Oh O., Agrawal M. and Rao H., 2011. Information control and terrorism: Tracking the Mumbai terrorist attack through twitter, Information
Systems Frontiers 13, pp. 33-43
Potts C., 2011. Potts Twitter-aware Tokeniser - http://sentiment.christopherpotts.net/code-data/happyfuntokenizing.py, [last viewed 29.3.2013]
Plutchik R., 1980. Emotion: A Psychoevolutionary Synthesis. Longman Higher Education.
Ritter A., Clark S., Mausam and Etzioni O., 2011. Named Entity Recognition in Tweets: An Experimental Study, Proceedings of Conference on
Empirical Methods in Natural Language Processing, Edinburgh (UK)
Sykora M., Jackson T. W., O’Brien A. and Elayan S., 2013. EMOTIVE Ontology: Extracting Fine-Grained Emotions from Terse, Informal
Messages, IADIS International Conference on Intelligent Systems and Agents 2013, Prague (Czech Republic).
Thelwall M., Buckley K. and Paltoglou G., 2012. Sentiment Strength Detection for the Social Web, Journal of the American Society for
Information Science and Technology 63, pp. 163-173
Tjong E. F., Sang K., Meulder F. D., Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition,
Proceedings of the ACL seventh conference on Natural Language Learning.