Serendipitous Recommendations
via Innovators
Noriaki Kawamae
SIGIR ’10
Summarized and presented by Sang-il Song
Page 1
Contents
•
•
Introduction
Related Work
–
Novelty Metric
– Novelty vs. Serendipity
•
Key Idea
– New serendipity measure based on time
– Innovative consumers
– User Flow
• Detail Model
– Personal Innovator Probability (PIP)
– User Flow Probability (UFP)
– Combination of PIP and UFP
• Experiments
– Datasets
– Comparison of top-N precision
– Comparison of other metrics
• Discussion
• Conclusion
Page 2
INTRODUCTION
• Traditional Collaborative Filtering
– Focuses on optimizing accuracy measurements such as precision and recall
– Neglects other factors such as novelty and serendipity
• Popularity Bias
– The most popular items in any log collection are the ones that a given user
will recognize with high probability, or be broadly familiar with.
– Highly accurate recommendations appear far too obvious and of little help to
users, and seem to fail to enhance user satisfaction
– Recommendation systems that appropriately discount popularity lead to an
increase in the total sales
• Novel and serendipitous recommendations
– Broaden user’s view in the recommendation flows
– Focus on the surprise of each user with the recommendation
Page 3
Related Work
• Novelty and serendipity metrics
– Measuring the Degree of “non-obviousness” of recommendations
• Interest-based Recommendation (Yang et al., 2005)
– Define novelty in terms of user knowledge and his/her degree of interest model
• Topic diversification approach (Ziegler et al., 2005)
– Topical diversification approach to balance and diversify personalized recommendat
ion lists
• Random Walk based Techniques (Focuss et al. 2007)
– Computing similarity of nodes solving sparsity problem of data
• Item centric evaluation for recommendation (Celma et al. 2008)
– User-centric evaluation aims to measure the perceived quality of the recommendati
ons
• These works don’t consider “dynamics”
– The changes in user preferences over time
– New items are different from old items in terms of their serendipity
Page 4
Novelty vs. Serendipity
• Distinction between novelty and serendipity
• Example
– There is a user whose favorite director “James Cameron”
– If the system recommends his latest movie “Avatar (2009)”
• Certainly novel but probably not serendipitous
– If the systems shows “Princess Mononoke (1997)”, which has them
es similar to “Avatar”
• Non-obvious recommendation
• Providing serendipity to the user
• Take much longer for find it by themselves in the absence of any help
Page 5
Key Idea 1 - How to quantify serendipity?
• “Time” can be used to quantify serendipity?
– The difference between the time at which target user would purchase ite
ms and the time at which user these items are recommended to user
– In the below figure
• | tˆa3 - t0 | indicates a longer acquisition delay than | tˆa2 - t0 |
• i3 seems to offer more serendipity to ua than i2
Page 6
Key Idea 2 - “Innovative” consumers
• “Innovative” Consumers
– The early consumers who focus on undiscovered or unreleased items are
called “innovative”
– Innovators become aware of items well before their release
– The purchase logs of innovators include more serendipitous items that oth
er like-minded consumers would like to acquire but have not yet become
aware of
• Innovators are able to discover even poorly marketed items
– These items, once “discovered”, are rapidly acquired by like-minded follow
ers
• Innovative consumer is captured into recommendation model u
sing “Personal Innovator Probability (PIP)”
Page 7
Key Idea 3 - User Flow
• Traditional algorithms based on item-item similarity compute the simil
arity scores among items using the number of consumers who purcha
sed both items
– Assumption: “Those who agreed in terms of past behavior will tend to agree in the
future”
• In the below figure
– Each item of set i1, i2, i3 has been purchased by same users in common with ia
– Each item is treaded equally according to the similarity defined for ia in traditional
system
– There needs to differentiate these items by focusing on the order of item purchase
time
– i3 is more appropriate item for a user who has purchased ia than i1 or i2
• This property is modeled as User Flow Probability (UFP)
Page 8
Personal Innovator Degree
• Personal Innovator Degree (PID)
– Weight user logs appropriately given a target user and to identify logs tha
t match the precedence preference of the target user
– Two time factors and one item factor
PID(ua , ub ) =
å r(i;b, a) ´ w(i;a)´ v(i)
iÎCab
Cab
r(i;b, a)
w(i;a)
v(i)
Innovator
Recency
Discriminativity
The set of common items that both ua and ub ha
ve purchased
The degree of ub as a personal innovator to ua
for common item i
How recently ua purchased common item i
How important common item i is as a discrimina
tor of like-minded users
Page 9
Personal Innovator Degree Detail
• Degree of innovator
– A
– Smaller tb,i and bigger ta,i make Degree of innovator bigger
• Recency
–
ea,i
w(i;a) = exp(- )
ea
Degree of Innovator
B가 A보다 아이템을 먼저 구매했을 경우 ➜ 커진다
B보다 A가 먼저 아이템을 구매했을 경우 ➜ 0
– Smaller elapsed time passed since ua purchased j makes it bigger
• Degree of discriminator
Recency
아이템이 나오자마자 구매한 경우 ➜ 1
아이템이 나온지 한참후에 구매한 경우 ➜ 0에 가까움
1
log(1+Ui )
–
v(i) =
–
Unpopular item is important discriminator
Degree of discriminator
Unpopular한 아이템 ➜ 유저의 특성 파악에 유용한 정보 ➜ 큰 값
Popular한 아이템 ➜ 유저의 특성 파악에 의미 없는 정보 ➜ 0에 가까움
Page 10
Personal Innovator Probability
• Personal Innovator Probability
– Improved version of Personal innovator Degree
– Solve the sparsity problem of data
• The ratio of users who share common items decreases rapidly in inverse propo
rtion to the number of times
• Innovator probability
– Using Markov Chain (Random Walk)
– Nodes are corresponding to user and edge is p(b | a) =
PID(ub, ua )
å PID(u, ua )
u
– Normalized PID over all users
–
eeT
P = a P +(1- a )
U
– Ṗ is ergodic Markov chain
–
P = e- b ((b P) +
1
(b P)2 +
2!
모두 유저간 edge에 대하여 (1-a)/|U|의 weight를 더해줌으
로써 Markov Chain Process가 수렴함을 보장
Ergodic Markov Chain
어떤 state에서 다른 어떠한 state로 이동이 가능
Random Walk 시 수렴 보장
+
1
(b P)N-1 + )
(N -1)!
N
– Weighted Hitting Time (weight: b )
N!
Page 11
Continuous Time Markov Chain
• Markov Chain
– random process endowed with the Markov property
– the next state depends only on the current state and
not on the past
• Continuous-Time Markov Chain
– Continuous parameter space (cf. discrete parameter space)
– Characterized by transition rate qij between state i and j
• How quickly that i➝j transition happens
Pr(i ® j) = qij ,
i¹ j
– Jump Rate: The rate that stochastic process leaves item i
qi = å qij
j¹i
Page 12
Distribution of the number of users who rated a DVD at a specific age of the DVD
X-axis: the number of users
Y-axis: the age of movies
Distribution of the number of users who rated a DVD at a specific age of the DVD in Netflix
• The fitting lines is exponential functions of time
• The number of user who evaluated an item decreases exponentially as
the item ages
Page 13
User Flow Probability
• The transition probability
– The probability that transition from item i to item j occurs
pij = å qi exp(-qi t u,ij )
t=0 일 때, pij = qi
t=∞ 일 때, pij = 0
t=0~∞사이에서 transition이 일어날 확률: 1
User들이 아이템 i를 선택한지 얼마 안되서 j를 선택한 경우,
pij가 커진다
u
• User Flow Probability
– Representing relationships of items
– Continuous-time Markov Chain
– UFP is computed using Taylor approximation
U(t f ) =
tf
n
(tQ)
ò ån=0 n! dt
0
¥
Continuous-time Markov Chain 모델 사용 (PIP와 유사)
단, hitting time이 시간에 대한 함수
특정 시점 tf에서의 trend는 0~tf까지 hitting 의 적분값
UFP(ib,ia ) = lim u(ib | ia , t f )
t f ®¥
UFP는 특정 시점이 아닌 모든 시간의 평균적인 trend
를 의미
Page 14
Recommendation based on both PIP and UFP
• Personal Innovator Probability
– the relationship between users
– The probability of innovator of user
• User Flow Probability
– Presenting relationship between items
• Combination Model
p(ib | ua ) µ åUFP(ib ,ia )d (ia | u j )PIP(u j , ua )
j
– δ(i,u) represents the evidence that u has purchased item i
• 1 if I is in the purchased history
• 0 otherwise
• Computational Cost
User kNN의 관점에서 보면
1. Similarity measure로 PIP 사용
2. Item rating의 aggregation에서 weight를 UFP 줌
– PIP, UFP can be pre-computed on all user-user pair and item-item pair
– The recommendation algorithm is computed in the linear time
Page 15
Experiment - Dataset
• Dataset
– purchase log data of on-line music download services
• 2005. 04. 01 ~ 2006. 07. 31
• #user: 84620
• #item: 44527
– Purchase log data of on-line video download service
• 2005. 09. 01 ~ 2006. 2. 28
• #user: 7537
• #item: 4064
– Netflix
•
•
•
•
•
1999. 11. 11 ~ 2005. 12 .31
#user: 480,189
#item: 17,770
#rating: 100,480,507
Binary rating (1 if purchase)
– Query log of search engine
• 2006. 04. 01 ~ 2006. 05. 31
• 33,325,842 records
• <user id, query keyword id, timestamp>
Page 16
Comparison of top-N precision
Top-10 Precision
0.5
0.45
Popular
0.4
Cosine
0.35
Pearson
Item
0.3
bPLSA
0.25
0.2
MEA
0.15
EABIF
PID
0.1
PIP
0.05
PIP+UFP
0
Video
Music
NetFlix
- Popular
- Cosine
User-based KNN with Cosine Similarity
- Pearson
User-based KNN with Pearson coefficient
- Item
Item-based KNN (2001)
- bPLSA
Probabilistic Latent Semantic Analysis wit
h Bernoulli Distribution (2003)
- MEA
Maximum Entropy Approach (2002)
- EABIF
Early-Adoption-Based information Flow m
ethod (2006)
- PID
Personal Innovator Degree (2009)
- PIP+UFP
Query
• PIP+UFP and PID shows higher precision than other recommendation
algorithms
–
–
–
–
Video: 0.4612 and 0.4602
Music: 0.3987 and 0.3922
Netflix: 0.1804 and 0.1793
Query: 0.1523 and 0.1588
Page 17
Comparison with Other Metric (1)
• Item Coverage (IC)
– The ratio of the number of unique items appearing in the top-N recomme
ndation list from the total number of items
– Measure of the item domain in the system from which the system can rec
ommend
• Average Elapsed Time (AE)
– How much time has passed from each item’s release day to its day of pur
chase by each user
– A short elapsed time indicates that the item is novel
• Average Difference Time (AD)
– The Difference observed time when each user u purchased item I in the te
st data without recommendation and the time of test
Page 18
Comparison with Other Metric (2)
•
Gini Coefficient
– A measure of the statistical dispersion of the distribution of users over items
– defined as the ratio of the areas on the Lorenz curve diagram
– Its value is between 0 and 1
•
•
•
•
High Gini coefficient indicates the distribution is extremely biased
0 corresponds to perfect equality
1 corresponds to perfect inequality
High Gini coefficient means that a few particular items tend to be ranked highly by most u
sers and thus recommendations are not strongly personalized
Page 19
Comparison with Other Metric (3)
Item Coverage
Average Elapsed Time
0.4
Popular
Cosine
0.3
50
Popular
Cosine
40
Pearson
30
Item
0.2
bPLSA
MEA
0.1
Pearson
EABIF
Item
bPLSA
20
MEA
EABIF
10
PID
0
Music
Video
NetFlix
Query
PID
0
PIP
Music
Average Distance Time
20
10
5
NetFlix
Query
1
Popular
Cosine 0.8
Cosine
Pearson
0.6
Item
Pearson
bPLSA 0.4
bPLSA
MEA
MEA
Item
0.2
EABIF
0
Music
Video
NetFlix
Query
PIP
Gini Coefficient
Popular
15
Video
PID
EABIF
0
Music
Video
NetFlix
Query
PID
Page 20
Discussion
• Effect of Innovator
– PIP improves AD
– Support that innovators know more serendipitous items than others
• PID vs PIP
– PID considers only pairwise comparison
– PIP uses an Markov chain to model how innovators are followed through
multiple steps
– PIP will recommend surprisingly interesting items that might not otherwis
e be discovered
• Novel Item Recommendation
– PIP shows shortest elapsed time
– It is best at identifying novel items
Page 21
Conclusion
• Contribution
– Impact of innovator of user (PIP)
– Impact of trends on the transition probability of items (UFP)
– New Metric for measuring serendipitousness
• Estimated time offset between recommendation time and purchase ti
me
• Future Work
– Incorporating user speciality and level of expertise in a particular domain
– Extending model to personalized information retrieval more generally
• It considers binary rating only (purchase or not)
• There needs comparison to state-of-the-art recommendation algorith
m
Page 22