Recsys12 && KDD 12 Brief Summary
Xiwang Yang
1
the 6th ACM International Conference on
Recommender Systems (RecSys 2012)
 RecSys 2012







2
Conference Introduction
Evaluation Metrics
Learning to Rank
Social RS
Context-aware and location-based recommendations
HCI, User-centric, interfaces & explanations
System Design
RecSys12 Introduction
 Premier global forum for discussing the state of the
art in recommender System
 Long paper acceptance rate: 24/119 = 20%
 Single Track
 > 270 attendence
 1/3 from industry
3
Evaluation Metrics
 Workshop:Recommendation Utility Evaluation: Beyond RMSE
 Organizer: Xavier Amatriain (Netflix), Harald Steck (Netflix),
Pablo Castells (UAM), Arjen de Vries, and Christian Posse (LikedIn)
 Specific questions that the workshop aims to address
include the following:
4
 What are the unmet needs and challenges for evaluation in
the RS field? What changes would we like to see? How could
we speed up progress?
 What relevant recommendation utility and quality dimensions
should be cared for? How can they be captured and
measured?
 How can metrics be more clearly and/or formally related to
the task, contexts and goals for which a recommender
application is deployed?
 How should IR metrics be applied to recommendation tasks?
What aspects require adjustment or further clarification?
What further methodologies should we draw from other
disciplines (HCI, Machine Learning, etc.)?
Evaluation Metrics
 Workshop:Recommendation Utility Evaluation: Beyond RMSE
 Can we predict the success of a recommendation algorithm
with our offline experiments? What offline metrics
correlate better and under which conditions?
 What are the outreach and limitations of offline evaluation?
How can online and offline experiments complement each
other?
 What type of public datasets and benchmarks would we want
to have available, and how can they be built?
 How can the recommendation effect be traced on business
outcomes?
 How should the academic evaluation methodologies improve
their relevance and usefulness for industrial settings?
 How do we envision the evaluation of recommender systems
in the future?
5
Evaluation Metrics
 Industry Keynote: Ron Kohavi (Microsoft): Online Controlled
Experiments: Introduction, Learnings, and Humbling Statistics
 Ron Kohavi, General Manager Experimentation Platform, Microsoft
 Controlled experiments at Microsoft Bing, very good work, 2012
kdd paper;
6
7
Controlled Experiments in One Slide
 Concept is trivial
 Randomly split traffic between
two (or more) versions
• A (Control)
• B (Treatment)
 Collect metrics of interest
 Analyze
Must run statistical tests to confirm differences are not
due to chance
Best scientific way to prove causality, i.e., the changes in
metrics are caused by changes introduced in the
treatment(s)
Evaluation Metrics
 Session: Multi-Objective Recommendation and Human Factors
 Multiple Objective Optimization in Recommendation Systems
 Mario Rodriguez and others explain how they design LinkedIn
recommendations by optimizing to several objectives at once
(e.g. candidate that is good for the job + who is open to new
opportunities). They report results from an AB Test run on
LinkedIn
8
Evaluation Metrics
 Session: Multi-Objective Recommendation and Human
Factors
 Pareto-Efficient Hybridization for Multi-Objective
Recommender Systems
• Marco Tulio Ribeiro-Universidade Federal de Minas Gerais &
Zunnit Technologies
• The problem of combining recommendation algorithms grows
significantly harder when multiple objectives are considered
simultaneously.
• take the multi-objective a step further. In their case, they
optimize the system to not only be accurate, but also present
novel or diverse items.
9
Outline
 Learning to Rank
• Session: Top-N Recommendation
• Social top-k RS:
• Industry invited talk: Ralf Herbrich (Facebook):
Distributed, Real-Time Bayesian Learning in Online
Services
10
Learning to Rank
 Focus more on ranking than rating prediction NOW!
 Session: Top-N Recommendation
 CLiMF: Learning to Maximize Reciprocal Rank with
Collaborative Less-is-More Filtering
• Best paper, Yue Shi, Delft, intern at Telefonica
• Optimize Mean Reciprocal Rank (MRR) directly.
• MRR is a well-known information retrieval metric for
measuring the performance of top-k recommendations
 Similar work: "TFMAP: Optimizing MAP for top-n contextaware recommendation
• SIGIR 2012.
• Optimize Top-N, Mean Average Precision
• Uses tensor factorization to model implicit feedback data
(e.g.,purchases, clicks) with contextual information
• fast learning algorithm
11
Learning to Rank
 Focus more on ranking than rating prediction NOW!
 Session: Top-N Recommendation
 "Ranking with Non-Random Missing Ratings: Influence of
Popularity and Positivity on Evaluation Metrics
• An interesting study on the very important issue of
negative sampling, and popularity bias in learning to rank.
The paper discusses these effects on the AUC (Area
Under the Curve) measure.
12
Learning to Rank
 Session: Top-N Recommendation
 "Sparse Linear Methods with Side Information for
Top-N Recommendations“
• University of Minnesota in the Twin Cities
• multidimensional context-aware learning to rank
 Alternating Least Squares for Personalized Ranking
• Gravity R&D
• Dense math
• invited anyone not interested in Mathematics to leave the room
• proposed a computationally efficient ranking based
method RankALS that optimizes the original objective
function, without sampling.
 "On Top-k Recommendation Using Social Networks”
13
Learning to Rank
 "Real-Time Top-N Recommendation in Social Streams
 University of Hannover
 Focus on analyzing social streams(twitter) in real-time
for personalized topic recommendation and discovery.
 Industry invited talk: Ralf Herbrich (Facebook): Distributed,
Real-Time Bayesian Learning in Online Services
 Bayesian Factor Models for large-scale distributed ranking
 The same author and others from MSR named it as
"Matchbox“, is now used in different settings
 Poster "The Xbox Recommendation System“
 Apply matchbox to recommending movies and games for the Xbox
 Poster “Collaborative Learning of Preference Rankings”
 Erasmus School of Economics & MSR
 apply it to sushi recommendation
14
Social RS
 Session: Social Recommendation
 "Spotting Trends: The Wisdom of the Few”
 Wisdom of the Few, using a reduced set of experts for
recommendations
 Popular != Trending
 iCoolhunt users are encouraged to take pictures of
objects that they think ‘cool’, upload them and share
them with friends online.
15
Social RS
 Workshop on Recommender Systems and the Social Web
 Extending FolkRank with Content Data
 Leveraging Publication Metadata and Social Data into
FolkRank for Scientific Publication Recommendation
 Context Determines Content - An Approach to
Resource Recommendation in Folksonomies
 FReSET - An Evaluation Framework for Folksonomybased Recommender Systems
 Aggregating Content and Network Information to
Curate Twitter User Lists
 Online Dating Recommender Systems: The SplitComplex Number Approach
 Social Media-Driven News Personalization
 Trust-Based Local and Social Recommendation
16
Context-aware and location-based
recommendations
 Workshop: Personalizing the Local Mobile Experience
 Workshop on Context-Aware Recommender Systems
 Session: Contextual and Semantically Aware
Recommendation
 Context-Aware Music Recommendation Based on Latent
Topic Sequential Patterns",
• playlist generation
 "Ads and the City: Considering Geographic Distance
Goes a Long Way”
• location-aware recommendations.
17
HCI, User-centric, interfaces & explanations
 Tutorial: Conducting User Experiments in Recommender
Systems
• Bart Knijnenburg, UCI
• overview of how to conduct user studies for recommender
systems
 Paper: TasteWeights: A Visual Interactive Hybrid
Recommender System
• USSB
 Paper: Inspectability and Control in Social
Recommenders
• Bart Knijnenburg, UCI
• Analyze the effect of giving more information and control
to users in the context of social recommendations.
 Workshop on Interfaces for Recommender System.
18
System Design
 Tutorial: Building Industrial-scale Real-world
Recommender Systems
• Xavier Amatriain, Netflix
 Mendeley Suggest: Engineering a Personalized Article
Recommendation System
• Kris Jack from Mendeley
• he explained how they make use of AWS and Mahout in a
system that can generate personalized recommendations
for about $60 a month
 From a toolkit of recommendation algorithms into a real
business
• Domonkos Tikk from Gravity R&D
• evolved from being a team in the Netflix Prize to a realworld company with very interesting projects
19
RecSys in China- Booming
 RecSys China
 Technical Community, ~5000 members
 RecSys 13 in HongKong
 Baidu
• RecSys team built two years ago, size: ~100
 Taobao
• RecSys team: > 100
 Weibo
• RecSys team: ~20
 Tencent
 Huawei
20
• Noah's Ark Research Lab in Hong Kong
• Built July, 2012
• long term ~100 researchers
How user evaluate each other in social media
 Keynote-Jure Leskovec-Stanford University
21
22
23
KDD2012
 August12-16, Beijing, China
 First held in Asian
 Attendance: 1232
 Long Paper Acceptance Rate: 133/755 = 17.6%
 Three Research Tracks about Recommendation
 Research Session : Personalization and
Recommendation
 Research Session: Ads and Video Recommendation
 Research Session: Recommendation
 Research Session : Matrices and Tensors
24
Mining Heterogeneous Information
Networks- Jiawei Han - UIUC
25
26
27
28
29
30
31
32
33
Big Data Panel Discussion
 debate on the following questions:
 What is the nature of Big Data? What are the Big Data
problems that you have encountered? Is this a longterm challenge or a short-term fad?
 What opportunities and challenges does data mining
face on Big Data?
 What are effective Big Data solutions? What
platforms, sampling solutions, and applications are most
effective for handling Big Data?
34
Some of the Opinions
 Christos Faloutsos – CMU
 Large data size: > 100 machines, > 1 Tera Bytes
 Jiawei Han
 Large Complexity
 Michael I. Jordan
 ? Sampling everything
 Some other voice
 Big data rate in HFT
35