Identifying and Incorporating
Latencies in Distributed Data Mining
Algorithms
Michael Sevilla
Identifying and Incorporating
Latencies in Distributed Data Mining
Algorithms
Michael Sevilla
Applicability of Mahout for Large
Data Sets
Michael Sevilla
What is Mahout?
• Distributed machine learning libraries
– “scalable to reasonably large data sets”
– Runs on Hadoop
http://heureka.blogetery.com/
The Data: Million Song Data Set
• Large Data Set
– 1,019,318 users
– 384,546 MSD songs
– 48,373,586 (user, song, count)
• Kaggle Competition: offline evaluation
– Predict songs a user will listen to using
• Training: 1M user listening history
• Validation: 110K users
• “Martin L” blogged his methodology + results
Motivations
•
•
•
•
Can Mahout easily be modified?
Can Mahout perform well for this workload?
Can Mahout produce accurate results?
Can Mahout work ‘out of box’?
• Hypothesis: 22 machines + Mahout > 1 guy
´
22
vs.
What kind of Recommender?
• Format: <userID, songID, count>
• Users interacting with items
• Users express preferences towards items
• We can us Collaborative Filtering
´
22
vs.
Collaborative Filtering
• Predicts preference of user towards an item
• Constructs a Top-N-Recommendation
1. Parse input training data
2. Create user-item-matrix
3. Predict missing entries
é x 00
ê
ê x 01
ê
ê
ê
êë x 0 m
x10
xij
xn 0 ù
ú
ú
ú
ú
ú
xnm úû
Mahout has item-based Collaborative Filtering jobs!
CAN MAHOUT EASILY BE MODIFIED?
Martin’s Code
• Methodology: similarity vector of history
– Sparse-matrix
• COLISTEN(i, j) – listeners who listened to i and j
– Sum similarities for each song user x listens to
• The code: all python
– Parse: 27 lines of code (l.o.c)
– Create Matrix: 46 l.o.c
– Predict: 45 l.o.c
åCOLISTEN[i,:]
é x 00
ê
ê x 01
COLISTEN(i, j) = ê
ê
ê
êë x 0 m
x10
xij
xn 0 ù
ú
ú
ú
ú
ú
xnm úû
Mahout’s Code
• Methodology:
– No Idea…
• The code: all java
– Poorly commented
– 14 *.java files
– Many Directories
• ~/mahout/core/src/main/java/org/apache/ma
hout/cf/taste/hadoop/item/RecommenderJob.
java
– RecommenderJob.java: 284 lines of code (l.o.c)
– SimilarityMatrixRowWrapperMapper.java: 47 l.o.c
– UserVectorSplitterMapper.java: 138 l.o.c
Mahout’s Code
CAN MAHOUT EASILY BE MODIFIED?
NO
CAN MAHOUT PERFORM WELL FOR
THIS WORKLOAD?
Martin’s Code
• Performance on 86MB:
– Parse data: 10 minutes
– Make Matrix: 22 minutes
– Predict songs for 11000 users: 1 hour, 18 minutes
• Did not test scalability
$/ python convertToNumbers.py
$/ python colisten.py
$/ python predict_colisten.py
Mahout’s Code
• Performance on 86MB:
– Parse Time: 10 minutes
– Total Time: 25 minutes
• Tested scalability
– 64MB, 128MB, 256MB, 1GB, 2GB, 3GB
Mahout’s Code
• Total Time
• ~ 12m, 43m, 1hr, 2hr, 4hr, >5hr ….
10 Nodes Failed
Mahout’s Code
• Prepare Jobs (parse): seconds - minutes
Mahout’s Code
• Recommend Jobs (predict): seconds - minutes
Mahout’s Code
• Create Matrix Jobs: minutes - hours
CAN MAHOUT PERFORM WELL FOR
THIS WORKLOAD?
NO
CAN MAHOUT PRODUCE ACCURATE
RESULTS?
Training Set
• Kaggle Million Song Subset: 110K users
– User 2: 16 entries – took out 8
– User 16: 32 entries – took out 8
– User 17: 25 entries – took out 8
å
Q
Martin’s Code
User 2:
User 16:
User 17:
MAP =
q=1
AveP(q)
Q
where Q is the number of queries
1 2 3
4
5
( + +
+
+
) / 9 = 0.074
3 7 127 283 401
1
2
3
( +
+
) / 9 = 0.005
67 120 314
1
( ) / 8 = 0.004
32
å
Q
Mahout’s Code
User 2:
User 16:
User 17:
MAP =
q=1
AveP(q)
Q
where Q is the number of queries
1 2
( + ) / 9 = 0.013
20 32
1 2
3
4
( + +
+
) / 9 = 0.010
28 94 162 268
1
( ) / 8 = 0.004
30
CAN MAHOUT PRODUCE ACCURATE
RESULTS?
YES
CAN MAHOUT WORK ‘OUT OF BOX’?
YES… but not well
Conclusion
• Mahout did not scale well
• Mahout was not easy to learn
• Mahout was not easily modifiable
• For performance and efficiency, it is better to
– Understand the data set
– Understand data mining
– Understand the methodology