COLOR-ATTRIBUTES-RELATED
IMAGE RETRIEVAL
WEEK 4
Student: Kylie Gorman
Mentor: Yang Zhang
GMM AND FISHER VECTOR
CODE
STEP ONE: CONCATENATE THE FEATURE
MATRICES OF EACH IMAGE
STEP TWO: APPLY GMM FUNCTION

Generate mean, covariance, and prior mode
probabilities
Mean
Covariance
Prior Mode
Probabilities
STEP THREE: CREATE FISHER VECTORS
encoding = vl_fisher(new', means, covariances, priors);
REPEAT PROCESS
FINAL STEPS
Calculate feature matrix of each image, isolating
the object first
 Concatenate matrices
 Apply PCA function to preprocess data
 Multiply each individual feature matrix by result
 Concatenate output into 1 matrix
 Apply GMM function and obtain mean,
covariance, and prior mode probabilities
 Apply Fisher Vector to each individual result to
obtain vectors that are the same size
 Use those fisher vectors for 11 SVM’s (one for
each color)

COMPLETE STEPS



Using Ebay Data (omitting binary images)
Use all Google Data (from 30 to 100 images per color)
Increase cluster size in GMM from 10 to 128
SVM
LINEAR SVM
First tried it with libsvm code
 MATLAB Function: svmtrain (Training, Group)
 Training: Data to be processed (transpose matrix)
 Group: Specifies +1 or -1 data
 Use SVM for each color (black, blue, brown,
green, grey, orange, pink, purple, red, white,
yellow)
 Changed to fitcsvm(X,Y)

SVM TRAIN OUTPUT
FITCSVM OUTPUT
CLASSIFY DATA
MATLAB Function:
svmclassify(SVMStruct,Sample)
 Use SVMStruct from svmtrain (from each color)
 Sample: Concatenated Ebay Fisher Vectors
 Changed to predict(SVMModel, X)
 SVMModel from fitcsvm

SVM CLASSIFY OUTPUT
Column vector with the same number of rows as
Sample. Each entry (row) in Group represents the
class of the corresponding row of Sample.
PREDICT OUTPUT
Returns Label and
Score
CURRENT PROGRESS
CALCULATE PRECISION
Calculate 12 highest scores for each color, using
first column only
 Determine if each score is a correct match by
checking indices
 Calculate each color’s precision

FUTURE GOALS
Fix Binary function
 Try process with new data set


Data set available: Fahad Shahbaz Khan, Rao
Muhammad Anwer, Joost van de Weijer, Andrew D.
Bagdanov, Maria Vanrell, Antonio M. Lopez
Image retrieval test
 Object Detection
