CoFiRank : Maximum Margin
Matrix Factorization for
Collaborative Ranking
Markus Weimer, Alexandros Karatzoglou,
Quoc Viet Le and Alex Smola
NIPS’07
Idea
• Maximum Margin Matrix Factorization
• Structured Estimation for Ranking
• Bundle Method Solver
Collaborative Filtering
• Based on partial observed matrix to
predict unobserved entries
Matrix Factorization
• Low Rank Approximation
• SVD for fully observed Y
• Non-convex
Maximum Margin Matrix Factorization
• Trace norm+Hinge loss: Convex
• Semi-Definite Programming
Regularized Matrix Factorization
• Formulation
minimize
U,V
Alternating optimizing
L(UV , Ytrain) 


trUU
2
T
 trVV T

• Probabilistic Matrix Factorization (PMF)
L(UV , Ytrain)  UV  Ytrain 2
2
• CoFiRank ( , Ytrain)  1  NDCG( , Ytrain)
• Linear Convex Upper Bound
L( f , Ytrain) : max ( , Ytrain)  c, f  f

Non-Convex

Solved by linear programming
How to Compute Loss?
• Linear Convex Upper Bound
L( f , Ytrain) : max ( , Ytrain)  c, f  f

• Solved by Linear Programming
min
X
 C
i
j
i, j
X i, j
Can this explain in simple way?

Useful Links
• CoFiRank
http://www.cofirank.org
• MMMF
http://ttic.uchicago.edu/~nati/mmmf/
• MF
http://helikoid.si/mf/index.html
Famous Researchers in
Optimization
• Yurii Nesterov – “Introductory Lectures on Convex
Optimization: A Basic Course”
http://www.core.ucl.ac.be/~nesterov/
• Arkadi Nemirovski – “Efficient methods in convex
programming”
http://www2.isye.gatech.edu/~nemirovs/
• Stephen P. Boyd – “Convex Optimization”
http://www.stanford.edu/~boyd/
• Stephen J. Wright – “Numerical Optimization”
http://pages.cs.wisc.edu/~swright/
• Dimitri Bertsekas – “Nonlinear Programming”
http://web.mit.edu/dimitrib/www/home.html
Questions?
Normalized Discounted Cumulative
Gain (NDCG)
How to set c?
 ( , f ) : c, f
• ci is set decreasing,  is maximized with
respect toπ for argsort(f)
• ci =(i+1)-0.25
Convex Upper Bound
• Linear Convex Upper Bound
L( f , Ytrain) : max ( , Ytrain)  c, f  f


L( f ,Ytrain)  ( * ,Ytrain)  c, f  f  ( * ,Ytrain)
sort (a), sort (b)  a, b
Bundle Method
• General convex optimization solver with
tight convergence bound O(1/)
Bundle Method for CoFiRank