KDD-2012 Summer School, August 10, 2012, Beijing, China
Modeling Opinions and
Beyond in Social Media
Bing Liu
University Of Illinois at Chicago
liub@cs.uic.edu
Introduction

Why are opinions so important?



Opinions are key influencers of our behaviors.
Our beliefs and perceptions of reality are
conditioned on how others see the world.
Whenever we need to make a decision we often
seek out others’ opinions.


True for both individuals and organizations
It is simply the “human nature”


We want to express our opinions
We also want to hear others’ opinions
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
2
Topics of this lecture

Sentiment analysis and opinion mining



It has been studied extensively in the past 10 years. A
large number of applications have been deployed.
We will define/model this task and introduce some core
research and challenges.
Going beyond: comments, discussions/debates

Beyond expressing our opinions in isolation, we also like
to comment, argue, discuss and debate.



They involve user interactions.
These are opinions too but of a slightly different type
We will try to model some of these interactive forums
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
3
Roadmap

Sentiment Analysis and Opinion Mining







Beyond Sentiments



Problem of Sentiment Analysis
Document sentiment classification
Sentence subjectivity & sentiment classification
Aspect-based sentiment analysis
Mining comparative opinions
Opinion spam detection
Modeling review comments
Modeling discussions/debates
Summary
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
4
Sentiment analysis and opinion mining

Sentiment analysis or opinion mining

computational study of opinions, sentiments,
appraisal, and emotions expressed in text.


Its inception and rapid growth coincide with
those of the social media on the Web


Reviews, blogs, discussions, microblogs, social networks
For the first time in human history, a huge volume
of opinionated data is recorded in digital forms.
A core technology for social media analysis

Because a key function of social media is for
people to express views & opinions
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
5
A fascinating and challenging problem!

Intellectually challenging & many applications.

A popular research topic in NLP, text and Web mining
(Edited book: Shanahan, Qu, & Wiebe, 2006; Book Chapters: Liu, 2007 & 2011; Surveys: Pang &
Lee 2008; Liu, 2012)

It has spread from computer science to management
science and social sciences (Hu, Pavlou & Zhang, 2006; Archak, Ghose &
Ipeirotis, 2007; Liu et al 2007; Park, Lee & Han, 2007; Dellarocas et al., 2007; Chen & Xie 2007).


Almost no research before early 2000.


> 350 companies working on it in USA.
Either from NLP or Linguistics (no data?)
Potentially a major technology from NLP.


But it is very hard!
People grossly underestimated the difficulty earlier.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
6
Roadmap

Sentiment Analysis and Opinion Mining







Beyond Sentiments



Sentiment Analysis Problem
Document sentiment classification
Sentence subjectivity & sentiment classification
Aspect-based sentiment analysis
Mining comparative opinions
Opinion spam detection
Modeling review comments
Modeling discussions/debates
Summary
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
7
Abstraction (1): what is an opinion?

Find a structure from the unstructured text.

Id: Abc123 on 5-1-2008 “I bought an iPhone a few days
ago. It is such a nice phone. The touch screen is really
cool. The voice quality is clear too. It is much better than
my old Blackberry. However, my mother was mad with me
as I did not tell her before I bought the phone. She also
thought the phone was too expensive, …”

One can look at this review/blog from



Document level, i.e., is this review + or -?
Sentence level, i.e., is each sentence + or -?
Entity and feature/aspect level
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
8
Entity and feature/aspect level

Id: Abc123 on 5-1-2008 “I bought an iPhone a few days
ago. It is such a nice phone. The touch screen is really
cool. The voice quality is clear too. It is much better than
my old Blackberry. However, my mother was mad with me
as I did not tell her before I bought the phone. She also
thought the phone was too expensive, …”

What do we see?




Opinion targets: entities and their features/aspects
Sentiments: positive and negative
Opinion holders: persons who hold opinions
Time: when opinions are given
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
9
Two main types of opinions
(Jindal and Liu 2006; Liu, 2010)

Regular opinions: Sentiment/opinion
expressions on some target entities

Direct opinions:


Indirect opinions:


“After taking the drug, my pain has gone.”
Comparative opinions: Comparisons of more
than one entity.


“The touch screen is really cool.”
E.g., “iPhone is better than Blackberry.”
We focus on regular opinions in this talk, and
just call them opinions.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
10
Basic Definition of an Opinion

Definition: An opinion is a quadruple,


(target, sentiment, holder, time)
This definition is concise, but is not easy to
use in many applications.



The target description can be quite complex.
E.g., “I bought a Canon G12 camera last week.
The picture quality is amazing.”
Target = picture quality? (not quite)
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
11
A More Practical Definition
(Hu and Liu 2004; Liu, in NLP handbook, 2010)

An opinion is a quintuple
(ej, ajk, soijkl, hi, tl),






ej is a target entity.
ajk is a feature/aspect of the entity ej.
soijkl is the sentiment value of the opinion of the
opinion holder hi on aspect ajk of entity ej at time tl.
soijkl is +ve, -ve, or neu, or a more granular rating.
hi is an opinion holder.
tl is the time when the opinion was expressed.
Still a simplified definition (see Liu, 2012 book)
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
12
Structure the unstructured

Objective: Given an opinion document,


Discover all quintuples (ej, ak, soijkl, hi, tl),
Or, solve some simpler forms of the problem


With the quintuples,




E.g., sentiment classification at the document or sentence
level.
Unstructured Text  Structured Data
Traditional data and visualization tools can be used to
slice, dice and visualize the results.
Enable qualitative and quantitative analysis.
The definition/model is widely used in industry
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
13
Abstraction (2): Opinion Summary

With a lot of opinions, a summary is necessary.


Different from traditional summary of facts


A multi-document summary task
1 fact = any number of the same fact
Opinion summary has a quantitative side


1 opinion  any number of the same opinion
The quintuple representation provides a basis for
opinion summarization.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
14
(Aspect)Feature-based opinion summary
(Hu & Liu, 2004)
““I bought an iPhone a few days
ago. It is such a nice phone. The
touch screen is really cool. The
voice quality is clear too. It is
much better than my old
Blackberry,. However, my
mother was mad with me as I did
not tell her before I bought the
phone. She also thought the
phone was too expensive, …”
….
Feature Based Summary of
iPhone:
Feature1: Touch screen
Positive: 212

The touch screen was really cool.

The touch screen was so easy to
use and can do amazing things.
…
Negative: 6

The screen is easily scratched.

I have a lot of difficulty in removing
finger marks from the touch screen.
…
Feature2: voice quality
…
Note: We omit opinion holders
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
15

Opinion observer - visualization (Liu et al. 05)
+
Summary of
reviews of
Cell Phone 1
_
Voice

Comparison of
reviews of
Screen
Battery
Size
Weight
+
Cell Phone 1
Cell Phone 2
_
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
16
Feature/aspect-based opinion summary
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
17
Google Product Search
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
18
Not just ONE problem

(ej, ajk, soijkl, hi, tl),









ej - a target entity: Named Entity Extraction (more)
ajk - a feature/aspect of ej: Information Extraction (more)
soijkl is sentiment: Sentiment Identification
hi is an opinion holder: Information/Data Extraction
tl is the time: Information/Data Extraction
Coreference resolution
Synonym match (voice = sound quality)
…
A multifaceted and integrated problem!
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
19
Roadmap

Sentiment Analysis and Opinion Mining







Beyond Sentiments



Sentiment Analysis Problem
Document sentiment classification
Sentence subjectivity & sentiment classification
Aspect-based sentiment analysis
Mining comparative opinions
Opinion spam detection
Modeling review comments
Modeling discussions/debates
Summary
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
20
Document sentiment classification

Classify a whole opinion document (e.g., a
review) based on the overall sentiment of the
opinion holder (Pang et al 2002; Turney 2002, …)



An example review:



Classes: Positive, negative (possibly neutral)
Neutral or no opinion is hard. Most papers ignore it.
“I bought an iPhone a few days ago. It is such a nice
phone, although a little large. The touch screen is cool.
The voice quality is clear too. I simply love it!”
Classification: positive or negative?
Classification methods: SVM, Naïve Bayes, etc
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
21
Assumption and goal

Assumption: The doc is written by a single person
and express opinion/sentiment on a single entity.

Goal: discover (_, _, so, _, _),
where e, a, h, and t are ignored

Reviews usually satisfy the assumption.



Almost all papers use reviews
Positive: 4 or 5 stars, negative: 1 or 2 stars
Forum postings and blogs do not


They can mention and compare multiple entities
Many such postings express no sentiments
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
22
Features for supervised learning

The problem has been studied by numerous
researchers

Probably the most extensive studied problem


Including domain adaption and cross-lingual, etc.
Key: feature engineering. A large set of features
have been tried by researchers. E.g.,





Terms frequency and different IR weighting schemes
Part of speech (POS) tags
Opinion words and phrases
Negations
Syntactic dependency, etc
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
23
Domain adaptation (transfer learning)

Sentiment classification is sensitive to the domain
of the training data.

A classifier trained using reviews from one domain often
performs poorly in another domain.



words and even language constructs used in different
domains for expressing opinions can be quite different.
same word in one domain may mean positive but negative
in another, e.g., “this vacuum cleaner really sucks.”
Existing research has used labeled data from one domain
and unlabeled data from the target domain and general
opinion words for learning (Aue and Gamon 2005; Blitzer et al
2007; Yang et al 2006; Pan et al 2010; Wu, Tan and Cheng 2009;
Bollegala, Weir and Carroll 2011; He, Lin and Alani 2011).
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
24
Cross-lingual sentiment classification

Useful in the following scenarios:



E.g., there are many English sentiment corpora, but for
other languages (e.g. Chinese), the annotated
sentiment corpora may be limited.
Utilizing English corpora for Chinese sentiment
classification can relieve the labeling burden.
Main approach: use available language corpora to train
sentiment classifiers for the target language data.
Machine translation is typically employed

(Banea et al 2008; Wan 2009; Wei and Pal 2010; Kim et al. 2010;
Guo et al 2010; Mihalcea & Wiebe 2010; Boyd-Graber and Resnik
2010; Banea et al 2010; Duh, Fujino & Nagata 2011; Lu et al 2011)
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
25
Roadmap

Sentiment Analysis and Opinion Mining







Beyond Sentiments



Sentiment Analysis Problem
Document sentiment classification
Sentence subjectivity & sentiment classification
Aspect-based sentiment analysis
Mining comparative opinions
Opinion spam detection
Modeling review comments
Modeling discussions/debates
Summary
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
26
Sentence subjectivity classification




Document-level sentiment classification is too coarse
for most applications.
We now move to the sentence level.
Much of the early work on sentence level analysis
focuses on identifying subjective sentences.
Subjectivity classification: classify a sentence into
one of the two classes (Wiebe et al 1999)


Objective and subjective.
Most techniques use supervised learning as well.

E.g., a naïve Bayesian classifier (Wiebe et al. 1999).
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
27
Sentence sentiment analysis

Usually consist of two steps

Subjectivity classification


Sentiment classification of subjective sentences


To identify subjective sentences
Into two classes, positive and negative
But bear in mind


Many objective sentences can imply sentiments
Many subjective sentences do not express
positive or negative sentiments/opinions

E.g.,”I believe he went home yesterday.”
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
28
Assumption


Assumption: Each sentence is written by a
single person and expresses a single positive
or negative opinion/sentiment.
True for simple sentences, e.g.,


“I like this car”
But not true for compound and “complex”
sentences, e.g.,


“I like the picture quality but battery life sucks.”
“Apple is doing very well in this lousy economy.”
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
29
Roadmap

Sentiment Analysis and Opinion Mining







Beyond Sentiments



Sentiment Analysis Problem
Document sentiment classification
Sentence subjectivity & sentiment classification
Aspect-based sentiment analysis
Mining comparative opinions
Opinion spam detection
Modeling review comments
Modeling discussions/debates
Summary
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
30
We need to go further

Sentiment classification at both the document
and sentence (or clause) levels are useful, but


They do not identify the targets of opinions, i.e.,



They do not find what people liked and disliked.
Entities and their aspects
Without knowing targets, opinions are of limited use.
We need to go to the entity and aspect level.


Aspect-based opinion mining and summarization (Hu
and Liu 2004).
We thus need the full opinion definition.
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
31
Recall an opinion is a quintuple

An opinion is a quintuple
(ej, ajk, soijkl, hi, tl),
where





ej is a target entity.
ajk is an aspect/feature of the entity ej.
soijkl is the sentiment value of the opinion of the
opinion holder hi on feature ajk of entity ej at time tl.
soijkl is +ve, -ve, or neu, or a more granular rating.
hi is an opinion holder.
tl is the time when the opinion is expressed.
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
32
Aspect-based sentiment analysis


Much of the research is based on online reviews
For reviews, aspect-based sentiment analysis
is easier because the entity (i.e., product name)
is usually known


Reviewers simply express positive and negative
opinions on different aspects of the entity.
For blogs, forum discussions, etc., it is harder:



both entity and aspects of entity are unknown,
there may also be many comparisons, and
there is also a lot of irrelevant information.
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
33
Aspect extraction


Goal: Given an opinion corpus, extract all aspects
A frequency-based approach (Hu and Liu, 2004):
nouns (NN) that are frequently talked about are
likely to be true aspects (called frequent aspects) .


Pruning based on part-of relations and Web search, e.g.,
“camera has” (Popescu and Etzioni, 2005).
Supervised learning, e.g., HMM and CRF
(conditional random fields) (Jin and Ho, 2009; Jakob and
Gurevych, 2010).

Using dependency parsing + “opinion has target”
(Hu and Liu 2004, Zhuang,Jing and Zhu, 2006; Qiu et al. 2009)
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
34
Extract Aspects & Opinion Words
(Qiu et al., 2011)


A double propagation (DP) approach proposed
Use dependency of opinions & features to
extract both features & opinion words.



It bootstraps using a set of seed opinion
words, but no feature seeds needed.


Knowing one helps find the other.
E.g., “The rooms are spacious”
Based on the dependency grammar.
It is a domain independent method!
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
35
Rules from dependency grammar
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
36
Aspect-sentiment statistical models

This direction of research is mainly based on
topic models:


pLSA: Probabilistic Latent Semantic Analysis (Hofmann 1999)
LDA: Latent Dirichlet allocation (Blei, Ng & Jordan, 2003;
Griffiths & Steyvers, 2003; 2004)

Topic models:



documents are mixtures of topics
a topic is a probability distribution over words.
A topic model is a document generative model
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
37
Aspect-sentiment model (Mei et al 2007)



This model is based on pLSA (Hofmann, 1999).
It builds a topic (aspect) model, a positive
sentiment model, and a negative sentiment
model.
A training data is used to build the initial models.



Training data: topic queries and associated positive
and negative sentences about the topics.
The learned models are then used as priors to
build the final models on the target data.
Solution: log likelihood and EM algorithm
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
38
Multi-Grain LDA to extract aspects
(Titov and McDonald, 2008a, 2008b)

Unlike a diverse document set used for traditional
topic modeling. All reviews for a product talk about
the same topics/aspects. It makes applying PLSA or
LDA in the traditional way problematic.

Multi-Grain LDA (MG-LDA) models global topics and
local topics (Titov and McDonald, 2008a).


Global topics are entities (based on reviews)
Local topics are aspects (based on local context, sliding
windows of review sentences)
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
39
Aspect-rating of short text (Lu et al 2009)

This work makes use of short phrases, head
terms (wh) and their modifiers (wm), i.e.





(wm, wh)
E.g., great shipping, excellent seller
Objective: (1) extract aspects and (2) compute
their ratings in each short comment.
It uses pLSA to extract and group aspects
It uses existing rating for the full post to help
determine aspect ratings.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
40
MaxEnt-LDA Hybrid (Zhao et al. 2010)
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
41
Graphical model

yd,s,n indicates




MaxEnt is used to
train a model
using training set



d,s,n
xd,s,n feature vector
ud,s,n indicates


Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
Background word
Aspect word, or
Opinion word
General or
Aspect-specific
42
Topic model of snippets
(Sauper, Haghighi and Barzilay, 2011)

This method works on short snippets already
extracted from reviews.


“battery life is the best I’ve found”
The model is a variation of LDA but with
seeds for sentiment words as priors,

but it also has HMM for modeling the sequence of
words with types (aspect word, sentiment word, or
background word).
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
43
Semi-supervised model
(Mukherjee and Liu, ACL-2012)


Unsupervised modeling is governed by “higherorder co-occurrence” (Heinrich, 2009), i.e.,
based on how often terms co-occur in different
contexts.
It results in not so “meaningful” clustering
because conceptually different terms can cooccur in related contexts e.g., in hotel domain
stain, shower, walls in aspect Maintenance; bed,
linens, pillows in aspect Cleanliness, are equally
probable of emission for any aspect.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
44
Semi-supervised model (contd.)

Semi-supervised modeling allows the user to
give some seed aspect expressions for a
subset of aspects (topic clusters)


In order to produce aspects that meet the user’s
need.
Employ seeds to “guide” model clustering,
not by “higher order co-occurrence” alone.

Standard multinomial => 2-level tree structured
priors
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
45
Graphical model
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
46
Aspect sentiment classification


For each aspect, identify the sentiment or opinion
expressed about it.
Classification based on sentence is insufficient. E.g.




“The battery life and picture quality are great (+), but the view
founder is small (-)”.
“Apple (+) is doing well in this bad economy (-).”
“Standard & Poor downgraded Greece's credit rating (-)”
Classification needs to consider target and thus to
segment each sentence


Lexicon-based approach (e.g., Ding, Liu and Yu, 2008)
Supervised learning (e.g., Jiang et al. 2011)
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
47
Aspect sentiment classification

Almost all approaches make use of opinion words and
phrases. But notice:



Some opinion words have context independent orientations,
e.g., “good” and “bad” (almost)
Some other words have context dependent orientations, e.g.,
“small” and “sucks” (+ve for vacuum cleaner)
Lexicon-based methods



Parsing is needed to deal with: Simple sentences,
compound sentences, comparative sentences,
conditional sentences, questions, etc
Negation (not), contrary (but), comparisons, etc.
A large opinion lexicon, context dependency, etc.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
48
A lexicon-based method (Ding, Liu and Yu 2008)



Input: A set of opinion words and phrases. A pair (a, s),
where a is an aspect and s is a sentence that contains a.
Output: whether the opinion on a in s is +ve, -ve, or neutral.
Two steps:
 Step 1: split the sentence if needed based on BUT words
(but, except that, etc).
 Step 2: work on the segment sf containing a. Let the set of
opinion words in sf be w1, .., wn. Sum up their orientations
(1, -1, 0), and assign the orientation to (a, s) accordingly.
wi .o
n
i 1 d (w , a)
i
where wi.o is the opinion orientation of wi. d(wi, a) is the
distance from a to wi.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
49
Sentiment shifters (e.g., Polanyi and Zaenen 2004)



Sentiment/opinion shifters (also called
valence shifters are words and phrases that
can shift or change opinion orientations.
Negation words like not, never, cannot, etc.,
are the most common type.
Many other words and phrases can also alter
opinion orientations. E.g., modal auxiliary
verbs (e.g., would, should, could, etc)

“The brake could be improved.”
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
50
Sentiment shifters (contd)

Some presuppositional items also can change
opinions, e.g., barely and hardly



Words like fail, omit, neglect behave similarly,


“This camera fails to impress me.”
Sarcasm changes orientation too


“It hardly works.” (comparing to “it works”)
It presupposes that better was expected.
“What a great car, it did not start the first day.”
Jia, Yu and Meng (2009) designed some rules
based on parsing to find the scope of negation.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
51
Basic rules of opinions
(Liu, 2010)

Opinions/sentiments are governed by many
rules, e.g.,

Opinion word or phrase, ex: “I love this car”
P
N

::=
::=
a positive opinion word or phrase
an negative opinion word or phrase
Desirable or undesirable facts, ex: “After my wife
and I slept on it for two weeks, I noticed a
mountain in the middle of the mattress”
P
N
::=
::=
desirable fact
undesirable fact
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
52
Basic rules of opinions

High, low, increased and decreased quantity of a
positive or negative potential item, ex: “The
battery life is long.”
PO
::=
|
NE ::=
|
NPI ::=
PPI ::=
no, low, less or decreased quantity of NPI
large, larger, or increased quantity of PPI
no, low, less, or decreased quantity of PPI
large, larger, or increased quantity of NPI
a negative potential item
a positive potential item
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
53
Basic rules of opinions

Decreased and increased quantity of an
opinionated item, ex: “This drug reduced my pain
significantly.”
PO
NE

::=
|
::=
|
less or decreased N
more or increased P
less or decreased P
more or increased N
Deviation from the desired value range: “This drug
increased my blood pressure to 200.”
PO
NE
::= within the desired value range
::= above or below the desired value range
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
54
Basic rules of opinions

Producing and consuming resources and wastes, ex:
“This washer uses a lot of water”
PO
NE
::=
|
|
|
::=
|
|
|
produce a large quantity of or more resource
produce no, little or less waste
consume no, little or less resource
consume a large quantity of or more waste
produce no, little or less resource
produce some or more waste
consume a large quantity of or more resource
consume no, little or less waste
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
55
Opinions implied by objective terms
(Zhang and Liu, 2011)


For opinion mining, many researchers first
identify subjective sentences and then
determine if they are positive/negative.
This approach can be problematic


Many objective sentences imply
opinions/sentiments
E.g., “After sleeping on the mattress for one month,
a valley is formed in the middle.”
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
56
Roadmap

Sentiment Analysis and Opinion Mining







Beyond Sentiments



Sentiment Analysis Problem
Document sentiment classification
Sentence subjectivity & sentiment classification
Aspect-based sentiment analysis
Mining comparative opinions
Opinion spam detection
Modeling review comments
Modeling discussions/debates
Summary
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
57
Comparative Opinions
(Jindal and Liu, 2006)

Gradable

Non-Equal Gradable: Relations of the type greater
or less than


Equative: Relations of the type equal to


Ex: “optics of camera A is better than that of camera
B”
Ex: “camera A and camera B both come in 7MP”
Superlative: Relations of the type greater or less
than all others

Ex: “camera A is the cheapest in market”
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
58
Analyzing Comparative Opinions

Objective: Given an opinionated document d,
Extract comparative opinions:
(E1, E2, F, po, h, t),
where E1 and E2 are the entity sets being
compared based on their shared features/aspects
F, po is the preferred object set of the opinion
holder h, and t is the time when the comparative
opinion is expressed.

Note: not positive or negative opinions.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
59
Roadmap

Sentiment Analysis and Opinion Mining







Beyond Sentiments



Sentiment Analysis Problem
Document sentiment classification
Sentence subjectivity & sentiment classification
Aspect-based sentiment analysis
Mining comparative opinions
Opinion spam detection
Modeling review comments
Modeling discussions/debates
Summary
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
60
Opinion Spam Detection
(Jindal et al, 2008, 2010 and 2011)
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
61
Supervised learning (fake reviews)
Training data
1.
2.
3.
4.




Same userid, same product
Different userid, same product
Same userid, different products
Different userid, different products
The last three types are very likely to be spam!
Other reviews, non-spam
Build a supervised classification model (Jindal
and Liu 2008)
(Ott et al., 2011) and (Li et al., 2011)
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
62
Finding Unexpected Behavior Patterns
(Jindal and Liu 2010)

Opinion spam is hard to detect because it is
very difficult to recognize fake reviews by
manually reading them.


Let us analyze the behavior of reviewers


i.e., hard to detect based on content
identifying unusual review patterns which may
represent suspicious behaviors of reviewers.
We formulate the problem as finding
unexpected rules and rule groups.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
63
Finding unexpected review patterns


For example, if a reviewer wrote all positive
reviews on products of a brand but all negative
reviews on a competing brand …
Finding unexpected rules,



Data: reviewer-id, brand-id, product-id, and a class.
Mining: class association rule mining
Finding unexpected rules and rule groups, i.e.,
showing atypical behaviors of reviewers.
Rule1: Reviewer-1, brand-1 -> positive (confid=100%)
Rule2: Reviewer-1, brand-2 -> negative (confid=100%)
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
64
The example (cont.)
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
65
Confidence unexpectedness
Rule: reviewer-1, brand-1  positive [sup = 0.1, conf = 1]
 If we find that on average reviewers give
brand-1 only 20% positive reviews
(expectation), then reviewer-1 is quite
unexpected.
Cu(v jk  ci ) 
E (Pr( ci | v jk , v gh )) 
Pr(ci | v jk )  E (Pr(ci | v jk ))
E (Pr(ci | v jk ))
Pr(ci | v jk ) Pr(ci | v gh )
m Pr( cr | v jk ) Pr( cr | v gh )
Pr(ci )r 1
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
Pr( cr )
66
Support unexpectedness
Rule: reviewer-1, product-1 -> positive [sup = 5]
 Each reviewer should write only one review
on a product and give it a positive (negative)
rating (expectation).
 This unexpectedness can detect those
reviewers who review the same product
multiple times, which is unexpected.


These reviewers are likely to be spammers.
Can be defined probabilistically as well.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
67
Detecting group opinion spam
(Mukherjee, Liu and Glance, WWW-2012)


A group of people who work together to
promote an product or to demote another
product.
The algorithm has two steps


Frequent pattern mining: find groups of people
who reviewed a number of products. These are
candidate spammer groups.
A relational model is then formulated to compute a
ranking of candidate groups based on their
likelihood being fake.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
68
Roadmap

Sentiment Analysis and Opinion Mining







Beyond Sentiments



Sentiment Analysis Problem
Document sentiment classification
Sentence subjectivity & sentiment classification
Aspect-based sentiment analysis
Mining comparative opinions
Opinion spam detection
Modeling review comments
Modeling discussions/debates
Summary
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
69
Modeling Review Comments
(Mukherjee and Liu, ACL-2012)



Online reviews by consumers evaluate products and
services that they have used.
While certainly useful, reviews only provide part of the
story: evaluations and experiences of the reviewers.
Hidden glitches:





Reviewer may not be an expert.
Misuses a product.
Doesn’t mention some product aspects of consumer interest.
Reviewer can be an opinion spammer writing fake reviews.
Clearly, there is a room for improvement of the online
review system.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
70
Review Comments



To improve the reviewing system, popular review
hosting sites (e.g., Amazon, Epinions, Wired.com,
etc.) support reader-comments on reviews.
Comments on review are a richer way of “review
profiling”, rather than just clicking whether the review
is helpful or not.
Many reviews receive a large number of comments.
(e.g., hundreds of them)


Reading them all to get a gist of them is not easy.
Some kind of summary will be very useful.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
71
What to model?

Topics/aspects and different types of
comments







Thumbs-up (e.g., “review helped me”)
Thumbs-down (e.g., “poor review”)
Question (e.g., “how to”)
Answer acknowledgement (e.g., “thank you for
clarifying”).
Disagreement (contention) (e.g., “I disagree”)
Agreement (e.g., “I agree”).
They are collectively called, C-expressions.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
72
Summary and usefulness

Extracted topics and C-expressions from
comments are quite useful in practice:



Enable more accurate classification of comments,
e.g., evaluating review quality and credibility.
Help identify key product aspects that people are
troubled with in disagreements and in questions.
Facilitate comments summarization. Summary may
include but not limited to:




% of people who giving a thumbs-up or thumbs-down
% of people who agree or disagree with the reviewer
Disagreed (contentious) aspects (or topics)
Aspects that people often have questions with
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
73
A graphical model – generative process
1.
2.
3.
For each C-expression type 𝑒, draw 𝜑𝑒𝐸 ~𝐷𝑖𝑟(𝛽𝐸 )
For each topic t, draw 𝜑𝑡𝑇 ~𝐷𝑖𝑟 𝛽𝑇
For each comment post 𝑑 ∈ {1 … 𝐷}:
i.
Draw 𝜃𝑑𝐸 ~𝐷𝑖𝑟 𝛼𝐸
𝑇
ii. Draw 𝜃𝑑 ~𝐷𝑖𝑟 𝛼 𝑇
iii. For each term 𝑤𝑑,𝑗 , 𝑗 ∈ {1 … 𝑁𝑑 }:
a.
Draw 𝜓𝑑,𝑗 ~𝑀𝑎𝑥𝐸𝑛𝑡 𝑥𝑑 , 𝑗
b.
Draw 𝑟𝑑,𝑗 ~𝐵𝑒𝑟𝑛𝑜𝑢𝑙𝑙𝑖(𝜓𝑑,𝑗 )
c.
if (𝑟𝑑,𝑗 = 𝑒 //𝑤𝑑,𝑗 is a C-expression term
Draw 𝑧𝑑,𝑗 ~ 𝑀𝑢𝑙𝑡(𝜃𝑑𝐸 )
else // 𝑟𝑑,𝑗 = 𝑡,𝑤𝑑,𝑗 is a topical term
Draw 𝑧𝑑,𝑗 ~ 𝑀𝑢𝑙𝑡(𝜃𝑑𝑇 )
𝑟
d.
𝑑,𝑗
Emit 𝑤𝑑,𝑗 ~ 𝑀𝑢𝑙𝑡(𝜑𝑧𝑑,𝑗
)
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
74
The graphical model in plate notation
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
75
Roadmap

Sentiment Analysis and Opinion Mining







Beyond Sentiments



Sentiment Analysis Problem
Document sentiment classification
Sentence subjectivity & sentiment classification
Aspect-based sentiment analysis
Mining comparative opinions
Opinion spam detection
Modeling review comments
Modeling discussions/debates
Summary
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
76
Modeling Online Discussions/Debates
(Mukherjee and Liu, KDD-2012)


A large part of social media is about discussion
and debate.
A large part of such contents is about social,
political and religious issues.


On such issues, there are often heated
discussions/debates, i.e., people argue and agree or
disagree with one another.
We can model such interactive social media.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
77
The Goal

Given a set of discussion/debate posts, we
aim to perform the following tasks.

Discover expressions often used to express



Contention/Disagreement (e.g., “I disagree”, “you
make no sense”) and
Agreement (e.g., “I agree”, “I think you’re right”). We
collectively call them CA-expressions.
Determine contentious topics.


First discover discussion topics in the whole collection,
then for each contentious post, discover the contention
points (or topics).
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
78
Joint modeling of debate topics and
expressions (JTE)

We jointly model topics and CA-expressions


Observation: A typical discussion/debate post
mentions a few topics (using semantically related
topical terms) and expresses some viewpoints with
one or more CA-expression types (using semantically
related contention and/or agreement expressions).
The above observation motivates the model

Posts are represented as random mixtures of latent
topics and CA-expression types.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
79
A graphical model – generative process
(the same as that for comments)
1.
2.
3.
For each C-expression type 𝑒, draw 𝜑𝑒𝐸 ~𝐷𝑖𝑟(𝛽𝐸 )
For each topic t, draw 𝜑𝑡𝑇 ~𝐷𝑖𝑟 𝛽𝑇
For each comment post 𝑑 ∈ {1 … 𝐷}:
i.
Draw 𝜃𝑑𝐸 ~𝐷𝑖𝑟 𝛼𝐸
𝑇
ii. Draw 𝜃𝑑 ~𝐷𝑖𝑟 𝛼 𝑇
iii. For each term 𝑤𝑑,𝑗 , 𝑗 ∈ {1 … 𝑁𝑑 }:
a.
Draw 𝜓𝑑,𝑗 ~𝑀𝑎𝑥𝐸𝑛𝑡 𝑥𝑑 , 𝑗
b.
Draw 𝑟𝑑,𝑗 ~𝐵𝑒𝑟𝑛𝑜𝑢𝑙𝑙𝑖(𝜓𝑑,𝑗 )
c.
if (𝑟𝑑,𝑗 = 𝑒 //𝑤𝑑,𝑗 is a C-expression term
Draw 𝑧𝑑,𝑗 ~ 𝑀𝑢𝑙𝑡(𝜃𝑑𝐸 )
else // 𝑟𝑑,𝑗 = 𝑡,𝑤𝑑,𝑗 is a topical term
Draw 𝑧𝑑,𝑗 ~ 𝑀𝑢𝑙𝑡(𝜃𝑑𝑇 )
𝑟
d.
𝑑,𝑗
Emit 𝑤𝑑,𝑗 ~ 𝑀𝑢𝑙𝑡(𝜑𝑧𝑑,𝑗
)
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
80
JTE in plate notation
(the same as that for comments)
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
81
JTE-R: Encoding reply relations



Observation: Whenever a post d replies to
the viewpoints of some other posts by
quoting them, and the posts quoted by d
should have similar topic distributions.
Let qd be the set of posts quoted by post d. qd
is observed.
Key challenge: - constrain 𝜃𝑑𝑇 to be similar to
𝜃𝑑𝑇 , where 𝑑 ∈ 𝑞𝑑 during inference while the
topic distributions of both 𝜃𝑑𝑇 and 𝜃𝑑𝑇 , 𝑑 ∈ 𝑞𝑑
are latent and unknown apriori.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
82
Exploiting Dirichlet distribution

A simple solution: exploit the following salient
features of the Dirichlet distribution:


𝑇
Since 𝜃𝑑𝑇 ~𝐷𝑖𝑟(𝛼 𝑇 ), we have 𝑡 𝜃𝑑,𝑡
= 1. Thus, it
suffices that 𝜃𝑑𝑇 can act as a base measure for
Dirichlet distributions of the same order.
Also, the expected probability mass associated with
each dimension of the Dirichlet distribution is
proportional to the corresponding component of its
base measure

𝐸 𝑋𝑖 =
𝛼
𝑖 .
𝛴𝛼
Thus, 𝐸 𝑋𝑖 ∝ 𝛼𝑖
𝑖
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
83
Exploiting Dirichlet distribution (contd)


We need functional base measures
Thus for posts that quote:


For posts that do not quote any other post,


we draw 𝜃𝑑𝑇 ~𝐷𝑖𝑟(𝛼 𝑇 𝒔𝒅 ), where 𝒔𝒅 = 𝑑′∈𝑞𝑑 𝜃𝑑𝑇′ |𝑞𝑑 |
(the expected topical distribution of posts in 𝑞𝑑 ).
we simply draw 𝜃𝑑𝑇 ~𝐷𝑖𝑟(𝛼 𝑇 ).
The Gibbs sampling is, however, an
approximation (see the paper for detail)
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
84
JTE-R in plate notation
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
85
JTE-P : Encoding Pair Structures

Observation: When authors reply to others’
viewpoints,



they typically direct their topical viewpoints with
contention or agreeing expressions to those
authors.
Such exchanges can go back and forth between
author pairs.
The discussion topics and CA-expressions
emitted are thus caused by the author-pairs’
topical interests and their nature of interactions.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
86
The approach



Let 𝑎𝑑 be the author of a post 𝑑, 𝑏𝑑 = [𝑏1…𝑛 ]
be the list of target authors to whom 𝑎𝑑
replies to or quotes in 𝑑.
The pairs of the form 𝑝 = (𝑎𝑑 , 𝑐 ), c ∈ 𝑏𝑑
essentially shapes both the topics and CAexpressions emitted in d as contention or
agreement on topical viewpoints are almost
always directed towards certain authors.
Thus, it is appropriate to condition 𝜃 𝑇 and
𝜃 𝐸 over author-pairs.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
87
The approach


To generate each term 𝑤𝑑,𝑗 , a target author,
𝑐~𝑈𝑛𝑖(𝑏𝑑 ), is chosen at uniform from 𝑏𝑑 forming
a pair 𝑝 = (𝑎𝑑 , 𝑐).
Then, depending on the switch variable 𝑟𝑑,𝑗 , a
topic or an expression type index 𝑧 is chosen
from a multinomial over topic distribution 𝜃𝑝𝑇 or
CA-expression type distribution 𝜃𝑝𝐸 , where the
subscript 𝑝 denotes the fact that the distributions
are specific to the author-target pair 𝑝 which
shape topics and CA-expressions.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
88
JTE-P graphical model
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
89
Roadmap

Sentiment Analysis and Opinion Mining







Beyond Sentiments



Sentiment Analysis Problem
Document sentiment classification
Sentence subjectivity & sentiment classification
Aspect-based sentiment analysis
Mining comparative opinions
Opinion spam detection
Modeling review comments
Modeling discussions/debates
Summary
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
90
Summary

We first introduced some basics of sentiment
analysis and opinion mining

Current solutions are still inaccurate.





Every sub-problem is hard
General NL understanding is probably hopeless in near future
But can we understand this restricted aspect of semantics?
Endless applications due to the human nature
We also discussed the problem of modeling
interactive social forums, such as review
comments and debates/discussions.

There is a lot of future work, e.g., linguistic knowledge.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
91
References
All references are in the
New Book
 Bing Liu. Sentiment Analysis
and Opinion Mining. Morgan
& Claypool Publishers. May
2012.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
92