HotOrNot Score Predictor
1. Motivation/ Introduction
2. Score Predictor
HotOrNot Score Predictor is a web-application that tries to predict the rating or score of the
facial image base on the rating form HotOrNot.com. We can divide our system into 3 phases
as shown in Fig1.
Figure 1: Overview of HotOrNot Score Predictor
2.1 Input Phase
In this part, we create crawler program to get thousands of images from
HotOrNot.com, which is one of the most popular rating site that allows users to rate the
attractiveness of photos submitted voluntarily by others. The images will be extracted from
the website with its score to show how attractiveness it is. Although, there are millions of
images in the website, some of them are inappropriate for our project. For example, some
images such as a girl in the bikini may have a higher score than another because of her body
not her face or some images that show less than 50% of their face. So, in the next step, we
try to eliminate those images from our system by using the photo filter. One technique that
we use in photo filter is skin detection. In this step, the images which have percent of the
skin greater than 70% of the whole image will be eliminated. After that, the result images
will be stored in the database as our input.
The next step in this phase is to extract the facial feature form those images and use
those feature as an input for the analysis phase. In this step, we call one of ‘Luxand’ library
which is called ‘Facial Feature Detection’ to extract the facial feature from the images. Using
the Luxand library, we get 40 attributes as show in the Figure 2 and the facial feature name
as show in Table 1
Figure 2: Example result image from Luxand facial feature detection
Facial Feature Name
Face Conture 1
Face Conture 2
Face Conture 3
Face Conture 4
Face Conture 5
Face Conture 6
Face Conture 7
Face Conture 8
Face Conture 9
Face Conture 10
Face Conture 11
Face Conture 12
Face Conture 13
Left Eye
Left Eye Inner Corner
Left Eye Outer Corner
Right Eye
Right Eye Inner Corner
Right Eye Outer Corner
Left Eye Brow Inner Corner
Facial Feature Name
Left Eye Brow Middle
Left Eye Brow Outer Corner
Right Eye Brow Inner Corner
Right Eye Brow Middle
Right Eye Brow Outer Corner
Nose Tip
Nose Bridge
Nose Left Wing
Nose Right Wing
Mouth Left Corner
Mouth Right Corner
Mouth Top
Mouth Bottom
Mouth Left Top
Mouth Right Top
Mouth Left Bottom
Mouth Right Bottom
Chin Bottom
Chin Left
Chin Right
Table 1: Facial Feature Name
We calculate and normalize the position of these features in the image as input data
for the analysis phase.
2.2 Analysis Phase
In this phase, we focus on one data mining approaches; Support Vector Machine (SVM) as a
classifier for our system.
Support Vector Machines (SVM) was invented by Vladimir Vapnik. It is a learning algorithm
for pattern classification which attempts to maximize the distance between the hyperplane
and the closest training samples on either side of the hyperplane.
Based on SVM principle, a linear SVM will find a linear function which separates the data into
two groups with the lowest misclassification as shown in Fig 3.
Figure 3: Linear SVM
In the other hand, the non-linear SVM will map linear non-separate data to a high
dimensional space where the data may be linearly separable as shown in Fig 4.
Figure 4: Nonlinear SVM
Given a labeled set of M training data (xi, yi) where xi is data and yi is the associated
classification label of xi. SVM will find the optimal hyperplane that correctly classifies the
largest fraction of training data while maximizing the distance of either class from the
hyperplane which is equivalent to minimizing the VC dimension within constructing an
optimal hyperplane. The discriminate hyperplane is defines as
𝑀
𝑓(𝑥) = ∑ 𝑦𝑖 𝛼𝑖 ∙ 𝑘(𝑥, 𝑥𝑖 ) + 𝑏
𝑖=1
where k() is a kernel function and the sign of f(x) determine the membership of x.
constructing an optimal hyperplane is equivalent to finding all nonzero 𝛼𝑖 Any vector xi that
corresponds to nonzero 𝛼𝑖 is a supported vector (SV).
For linear SVM, the kernel function is a dot product in the input space while the kernel
function in a nonlinear SVM projects the input to a higher dimension via a nonlinear
mapping function:
𝛷: 𝑅 𝑁 → 𝐹 𝑀 , 𝑀 ≫ 𝑁
then constructs a hyperplane in F. The motivation behind the mapping function is it is more
likely to find a linear hyperplane in the high dimensional space. Projecting input into the high
dimensional space can be defined as a kernel function that satisfying the condition
𝑘(𝑥, 𝑥𝑖 ) = Φ(𝑥) ∙ Φ(𝑥𝑖 )
Where 𝛷 is a nonlinear projection function.
The kernel that be used in this paper are given as follow:
1. Polynomial – degree p
𝐾(𝑥, 𝑦) = (𝑥 ∙ 𝑦 + 1)𝑝
2. Radial basis function
2 /2𝜎2
𝐾(𝑥, 𝑦) = 𝑒 −‖𝑥−𝑦‖
After we get extract 40 facial features from the first step, we use the corresponding
score and the 17 features of each image as input of ANN and SVM as show in the following
Facial Feature
Face Width
Left Eye Height
Right Eye Height
Left Eye Length
Right Eye Length
Distance Between Eyes
Nose Height
Nose Width
Nose Length
Chin Width
Left Eyebrow Length
Right Eyebrow Length
Distance Between Eyebrow
Left Eyebrow Height
Right Eyebrow Height
Mouth Length
Mouth Height
Meaning
max(Face Contour.x) - min(Face Contour.x)
Chin Left – Left Eye
Chin Right- Right Eye
Left Eye Inner Corner – Left Eye Outer Corner
Right Eye Inner Corner – Right Eye Outer Corner
Left Eye Inner Corner – Right Eye Inner Corner
Chin Bottom – Nose Tip
Nose Right Wing – Nose Left Wing
Nose Bridge – Nose Tip
Chin Right – Chin Left
Left Eyebrow Outer Corner – Left Eyebrow Inner Corner
Right Eyebrow Outer Corner– Right Eyebrow Inner Corner
Left Eyebrow Middle –Right Eyebrow Middle
Left Eyebrow Middle – Chin Left
Right Eyebrow Middle – Chin Right
Mouth Left Corner –Mouth Right Corner
Mouth Bottom – Chin Bottom
Table 2: Facial Feature and its meaning
Using SVM technique, we tested our training set in several kernel function and value to find
the appropriate function which has the highest classification correct rate. The function that
we used to test our training set are polynomial, with order 1, 2 and 3, and the Radial Basis
Function, with order 0.1, 1, 5 and 10. The result from our experiment has shown in Table 3.
Kernel Fn
value
Correct Rate (%)
Average
Polynomial
Radial Basis Function
1
92.05
78.26
57.81
81.47
77.40
2
91.08
79.21
57.68
80.60
77.14
3
92.45
78.12
57.43
80.47
77.12
0.1
92.05
78.11
57.45
80.52
77.03
1
92.08
78.26
57.68
80.47
77.12
10
92.53
79.01
57.32
80.21
77.27
Table 3: The correct rate from different kernel function and value
Each column here is the correct rate of each function. For example, if we use the linear
function to classify our data which have the real score 1-6.5 from the data which have the
real score more than 6.5, the correct rate is 92.05%.
As shown in the table, the correct rate of the first column and the last column are very high
because of there are only few images that have the score lower than 6.5 or equal to 10.
And the lowest correct rate of our function is around 57% which occurred when we classified
the images which have score equal to 8 from the images which have the score higher than 8.
This might be because most of the images have the score around 8-9 which is quite hard for
the SVM to find the optimal function that classifies the data into two groups with the lowest
misclassification.
Moreover, from the result here, you can see that even thought we change the order of the
polynomial function from 1-3 or we used the radius basis function with various values; the
correct rate doesn't change very much. That means our training set is linearly separated.
Then, we use the linear kernel as a kernel function in our system.
2.3 Output Phase
Giving an input image, our system will try to predict the score of the input image
based on the training set which we get from HotOrNot.com. This phase of the system isn’t
started yet, but we plan to start it as soon as we get the satisfying result from the analysis
phase.
3. Problems
In this project, we faced some problems. Our major problem is that about x% of the images
from HotOrNot.com is not suitable for our system. There are thousands of images in
HotOrNot.com which doesn’t show the face or doesn’t show the whole face of that person
as shown in Fig 5.
Although there is a face detector and feature extractor in our system, the detector doesn’t
work 100% correctly. There are still some errors form the detector as show in Fig 6.
Therefore, we have to spend some time to filter the input images by manually.
Moreover, the crawler has to score every image as 10 point to get that image from
HotOrNot.com. Sometime, these images are the new images which haven’t been rated by
anyone before.  Problem: incorrect score
4. Conclusion and Future Work
5. Acknowledgment