Event Detection in Twitter using Aggressive
Filtering and Hierarchical Tweet Clustering
Georgiana Ifrim, Bichen Shi, Igor Brigadir
Insight Centre for Data Analytics
University College Dublin
SNOW Data Challenge
April 8, 2014
Outline
Background
Method Proposed
Method Details
Results
Future Work
Background
Social media outlets (e.g., Twitter) play an increasing role in
the cycle of news production
Journalists use Twitter for news selection and presentation
Twitter:
• An endless, real-time, global stream of news
• Large scale and very noisy (redundant, messy content)
Challenge: Extract (close to real-time) newsworthy
topics/event/stories from the Twitter stream, in a format usable by
news professionals (e.g., topic-timestamp, topic-headline, topictags, tweet-ids, photo-urls)
Challenge
From this:
#Obama #follow #followme #followforfollow #followme #follower #followers
#alwaysfollowback #followbackalways #teamfollowback
I VOTED !!! #OBAMA http://instagram.com/p/RsoNuMgLkr/
@BarackObama #TeaamObama !!!! ✊🎋🎉🎊🇺🇸
Om 12u zou de eerste uitslag binnen zijn. Nu nog steeds niks. Dit trek ik niet. Wekker over
4u en we kijken dan wel. #obama #forward
My President is Black ★★★★★▄▄▄▄▄▄▄▄▄▄ ★★★★★▄▄▄▄▄▄▄▄▄▄
★★★★★▄▄▄▄▄▄▄▄▄▄ ▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄ ▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄ #Obama2012 #Retweet
voted at school! @barackobama I love you! #Forward2012 🇺🇸💙
http://instagr.am/p/RtONyRC9Tm/
Romney Romney Romney Romney Romney Romney Romney Romney!!!!!!!!!!!!!!!!!!!!!🇺🇸🇺🇸🇺🇸
🇺🇸 #RomneyRyan2012
Challenge
To this:
Obama wins Vermont
Romney wins Kentucky
Bernie Sanders wins Senate seat in Vermont
Romney wins Indiana
Method Proposed
1. Aggressive Data Filtering (to remove noise, to scale)
2. Hierarchical Clustering of Tweets + Dendrogram Cutting (to
obtain clusters without need of knowing #clusters a-priori)
3. Ranking of Clusters (to favor news-like topics)
4. Extracting Topic-Headlines (usable information)
5. Re-clustering Topic-Headlines (to remove topic fragmentation)
6. Extracting Final Topics (as presented to the user)
Method Details
Software
Collecting Twitter streams
SNOW Challenge Code (based on Twitter4J API)
All other development (https://github.com/heerme/twitter-topics)
Python2.7 + libraries: scipy, numpy, sklearn, nltk, json
Tweet-NLP: CMUTweetTagger (trained on tweets, entity detection)
Efficient clustering: fastcluster (C++ lib, interface to Python/R)
Data Collection
US Presidential Elections 2012
Collected from tweet ids (06-11-2012, 23:30 to 07-11-2012 06:51)
1,084,200 raw tweets text (english + non-english, 252MByte)
Syria, Ukraine, Bitcoin 2014
Collected from keywords + user ids (25-02-2014, 17:30 to 26-02-2014,
18:15)
1,088,593 raw tweets JSON (english + non-english, 4.3GByte)
943,175 english tweets JSON (3.8GByte)
943,175 tweets text (extract subset of fields from JSON object,
240MByte)
Replace re-tweet text with original tweet text
Data Pre-processing
Tweet filtering
Clean tweet-text. Remove: urls, user mentions, hashtags, punctuation, digits
Tokenize remaining text into tokens
Rebuild tweet by appending: user mentions (@) + hashtags (#) + text tokens
Remove tweets based on structure (remove if too many @, # or too few text tokens)
Term filtering
Keep only bi-grams + tri-grams occurring in at least a percentage of tweets in time
window (e.g., min(10, n_tweets_in_window * 0.0025))
Tweet-Term Matrix (binary)
Remove out-of-vocabulary tweets and very short tweets (with less than 5 tokens)
Retains about 20% of the original raw tweet stream (in each time window)
Hierarchical Tweet Clustering
Computing tweet pairwise-distance
Scale and normalize tweet-term matrix
Cosine as distance metric (euclidean similar results); sklearn + scipy
Computing hierarchical clustering => dendrogram
fastcluster C++ library (interface to R and Python)
Dendrogram cutting
Cut at 0.5 distance threshold (better libraries based on topology of
dendrogram available in R, e.g., Dynamic Tree Cut: only specify min
number of examples in each cluster)
One cluster = one potential topic
Hierarchical Tweet Clustering
Ranking clusters
Retain only clusters with at least 10 tweets (size constraint)
Score each cluster based on:
Compute cluster-centroid (vector of terms)
Get maximum term-score (over all centroid terms)
Term score: entity_score * burstiness_score
Assign the highest term score as cluster score
Normalize cluster score by cluster size
Entity score = 2.5 (identify entity-terms with CMUTweetTagger)
Burstiness score = df-idf_t, with t=4 (prior work on Bngram)
Interesting extensions to cluster score: article_score, tweet_importance
based on trustworthiness or clout of users issuing tweet
Hierarchical Tweet (Re)Clustering
Selecting topic-headlines
Take top-20 ranked clusters as potential topics
Select first (time-wise) tweet in each cluster as topic-headline
Re-cluster headlines
Hierarchical clustering of headlines
Score headline-clusters using max score headline
Rank headline-clusters, take top-10
Final topics
Select first (published) headline in each cluster, present raw tweet (less
url) to user
Gather all distinct keywords of headlines in headline-cluster to create
topic-tags
Tweet ids for topic: the ids of corresponding headlines. If headlines do
not cluster, only one tweet id
Results
Top-10 topics first time window in US stream (07-11-2012 00:00 – 00:10)
1. WASHINGTON (AP) - Obama wins Vermont; Romney wins Kentucky. #Election2012
2. Not a shocker NBC reporting #Romney wins Indiana & Kentucky #Obama wins Vermont
3. RT @SkyNewsBreak: Sky News projection: Romney wins Kentucky. #election2012
4. AP RACE CALL: Democrat Peter Shumlin wins governor race in Vermont. #Election2012
5. CNN Virginia exit poll: Obama 49\%, Romney 49\% #election2012
6. Mitt Romney Losing in Massachusetts a state that he governed. Why vote for him when
his own people don't want him? #Obama2012
7. Twitter is gonna be live and popping when Obama wins! #Obama2012
8. INDIANA RESULTS: Romney projected winner (via @NBC) #election2012
9. If Obama wins I'm going to celebrate... If Romney wins I'm going to watch Sesame Street
one last time #Obama2012
10. #election2012 important that Romney won INdependents in Virginia by 11 pts. With
parties about even, winning Inds is key
Results
Top-10 topics first time window in Syria stream (25-02-2014 18:00 – 18:15)
1. The new, full Godzilla trailer has roared online
2. At half-time Borussia Dortmund lead Zenit St Petersburg 2-0.
3. Ukraine Currency Hits Record Low Amid Uncertainty: Ukrainian currency, the hryvnia,
hits all-time low against ...
4. Ooh, my back! Why workers' aches pains are hurting the UK economy
5. Uganda: how campaigners are preparing to counter the anti-gay bill
6. JPost photographer snaps what must be the most inadvertantly hilarious political picture
of the decade
7. Fans gather outside Ghostbusters firehouse in N.Y.C. to pay tribute to Harold Ramis
8. Man survives a shooting because the Bible in his top pocket stopped two bullets
9. Ukraine's toppling craze reaches even legendary Russian commander, who fought
Napoleon
10. Newcastle City Hall. Impressive booking first from bottom on the left...
Discussion
Parameter choices
Filtering parameters dependent on window size (nr of tweets in window)
Unigrams vs N-grams (N>1)
Bi-grams + N-grams good for content + scalability
Cluster ranking
(Normalized) Df-idf_t seems a good choice, but cluster-score may benefit
from using tweet importance (based on user importance)
Topic Precision (~80%, based on googling topic-headlines)
On average about 8-9 out of 10 headlines are published news
Efficiency Aspect
System takes about 0.5min per 15min slot (scales well for larger time
slots)
Conclusion
Encouraging results in using Twitter stream as a news
aggregator (truly global)
Both sides now: media outlets (CNN, BBC, Reuters, AP) and
regular people post updates on (breaking) stories
We need a good topic-benchmark to refine techniques (e.g.,
comprehensive set of ground truth topics)
Future Work
Improve retrieval of newsworthy stories
-E.g., ‘This is what happens when you put two pit bulls in a photo booth’, vs
‘Ukraine currency hits record low amid uncertainty’
-May depend on type of stories we are after (BBC vs Sun)
-Tweet/user importance filtering may help
-News streamed in same time frame may help (vocabulary selection)
Fragmentation due to breaking news stories
-Same story discussed from different angles:
Lee Rigby murders: Michael Adebolajo given whole-life jail term
Lee Rigby murder sentence expected shortly. Pictured: the scene outside the Old Bailey in
London
Judge Mr Justice Sweeney says behaviour of Lee #Rigby's killers was "sickening and pitiless"
-Combination of tweet and term clustering may help (e.g., cluster headlines in
term rather than tweet space)
Thank You!
Open source code:
https://github.com/heerme/twitter-topics
Different Newspapers in UK
In the TV comedy seriesYes Minister, fictional Prime Minister
Jim Hacker explains to his staff the readership of the main
newspapers:
“The Daily Mirror is read by people who think they run the country,
The Guardian is read by people who think they ought to run the
country, The Times is read by people who actually do run the
country, The Daily Mail is read by the wives of the people who
run the country, The Financial Times is read by people who
own the country, The Morning Star is read by people who think
the country ought to be run by another country, and The Daily
Telegraph is read by people who think it is.”, Source: Wikipedia