Activity Summary
______________________________________________________________________________
1. Analyzing the Prediction vs. Actual Performance for the Known Set of Keys
Under this, we analyzed the performance of the models for the known set of keys. We performed
a few tests and analysis described below.
a. Prediction vs. Actual Plots for Median Averaging method
In this we cut the tails-off for the ensemble predictions (top 10 and bottom 10%) and then
find the median and average it with the two neighboring predictions. The final prediction
is plotted as is without rounding off. Then we checked the if we can increase the margin
to +/- 2 instead of +/- 1 as shown below in the figures. The first set is for Panelist 1, and
the second set is for Panelist 17.
Figure 1. Before and After plots for Panelist 1 (increasing the tolerance Margins)
Fugure 2. Before and After plots for Panelist 2
______________________________________________________________________
Evolutionary Design and Optimization Group
Activity Summary
______________________________________________________________________________
b. Prediction vs. Actual Plots for Majority Voting Method
In this after chopping off the tails we rounded each prediction and the final prediction is
what majority of them said is the liking score.
c. Model Use Frequency
d. Number of Models that come-up with the Actual liking Score Value for the known keys
2. Likelihood Density Models for Liking Score
We first generate 10000 random keys and apply our GP model ensembles for each panelist.
Distributions chosen:
1. Weibull: Is a parametric distribution with two parameters, called scale and shape. The Weibull
can imitate different distributions like normal, exponential and Rayleigh by varying its shape and
scale properties.
2. Normal: Just another name for Gaussian and has a mean and std as parameters.
3. KSDensity with Normal Kernel: This is a non-parametric and generates the entire PDF profile,
i.e., for each discrete value of liking score a probability. Matlab has this function called
"ksdensity".
Ensemble trimming method:
1. Throw away the bottom 10 and the top 10% of the ensemble predictions.
______________________________________________________________________
Evolutionary Design and Optimization Group
Activity Summary
______________________________________________________________________________
2. For Weibul estimation is valid for only greater than '0' values. So we check the <0 values and
take a percentage of these values to be 0.001 and keep them. The rest we throw them away. The
rationale behind saving a few is, may be these belong to real dislike.
The four methods are:
a. Weibul Fit -Per each key: For each key, we take the predictions from the entire ensemble and
estimate it to a Weibul. Save the scale and shape parameters. The final weibul is a mixture of all
these Weibuls.
b. WeibulFit -Entire Data: We merge the entire data for all the keys and then fit it to a weibul.
This gives us two parameters for the weibul.
c. KSDensity-Entire Data: We take the entire data for all the keys and apply the ksdensity and
plot the pdf. (Note: Doing this key wise resulted in the same pdf).
d. GaussianFit -Per each key: For each key, we take the predictions form the ensemble and
estimate it to Gaussian, Save the estimated parameters. The final Gaussian is the mixture of all
these Gaussians.
We formed all a, b, c, d methods and then chose b as our final modeling technique for density.
After we formed the distributions, we formed the clusters of Panelists into four different classes.
(i)
One that have all their mass between 0-1 and usually the bad panelists/bad models are
also in this group.
______________________________________________________________________
Evolutionary Design and Optimization Group
Activity Summary
______________________________________________________________________________
(ii)
Second the panelists whose majority of mass is between 0-3.5 (Hard to please)
______________________________________________________________________
Evolutionary Design and Optimization Group
Activity Summary
______________________________________________________________________________
(iii)
Panelists whose majority of mass is between 3.5 -6.5 (neutral)
(iv)
Panelists whose majority of mass is between 6.5 – 9.5 ( easy to please)
______________________________________________________________________
Evolutionary Design and Optimization Group
Activity Summary
______________________________________________________________________________
2. Serial Sensor Networks Modeling and Optimization
______________________________________________________________________
Evolutionary Design and Optimization Group
Activity Summary
______________________________________________________________________________
3. MOPSO Results
______________________________________________________________________
Evolutionary Design and Optimization Group