Color Compatibility From Large Datasets
Peter O’Donovan
Aseem Agarwala
Aaron Hertzmann
University of Toronto Adobe Systems, Inc. University of Toronto
Choosing colors is hard for many people
Choosing colors is hard for many people
?
How do designers choose colors?
How do designers choose colors?
Picasso
How do designers choose colors?
You the Designer
How do designers choose colors?
Krause [2002]
Goethe [1810]
Complementary Color Theory: colors opposite on the color wheel are compatible
Hue Templates: relative orientations producing compatible colors
Complementary
Monochromatic
Analogous
Triad
Color Harmonization
Cohen-Or et al. 2006
Photo and Video Quality Evaluation:
Focusing on the Subject
Luo and Tang 2008
Color Harmonization for Videos
Sawant and Mitra 2008
Aesthetic Visual Quality Assessment of Paintings
Li and Chen 2009
Adobe Kuler
527,935 themes
Ratings: 1-5 stars
Adobe Kuler
527,935 themes
Ratings: 1-5 stars
Adobe Kuler
527,935 themes
Ratings: 1-5 stars
Adobe Kuler
527,935 themes
Ratings: 1-5 stars
COLOURLovers
1,672,657 themes
Views and “Likes”
COLOURLovers
1,672,657 themes
Views and “Likes”
Goals
1. Analysis
Test hypotheses and compatibility models
2. Learn Models
Predict mean ratings for themes
3. New Applications
Develop new tools for choosing colors
Goals
1. Analysis
Test hypotheses and compatibility models
2. Learn Models
Predict mean ratings for themes
3. New Applications
Develop new tools for choosing colors
Kuler Dataset
104,426 themes
Ratings: 1-5 stars
COLOURLovers Dataset
383,938 themes
# Views and “Likes”
Mechanical Turk dataset
10,743 themes from Kuler
40 ratings per theme
1,301 total participants
Histogram of hue usage
% of all themes
Hue
Overall preference for warmer hues and cyans
Mean rating for themes containing a hue
Mean Rating
Hue
Overall preference for warmer hues and cyans
Histogram of hue adjacency (Kuler)
Histogram of hue adjacency (Kuler)
Histogram of hue adjacency (Kuler)
is more likely than
Histogram of hue adjacency (Kuler)
Significant structure
Histogram of hue adjacency (Kuler)
Significant structure
Warm hues pair well with each other
Histogram of hue adjacency (Kuler)
Significant structure
Warm hues pair well with each other
Greens and purples more compatible with themselves
Hue Template Analysis
Hue Templates: relative orientations producing compatible colors
Hue Templates: relative orientations producing compatible colors
Templates are rotationally invariant
Hue Templates: relative orientations producing compatible colors
Complementary
Monochromatic
Analogous
Different templates equally compatible
Triad
Hue adjacency in a theme (Kuler)
Diagonal lines are hue templates (Kuler interface bias)
Hue adjacency in a theme (Kuler)
Complementary template
Diagonal lines are hue templates (Kuler interface bias)
Hue adjacency in a theme (Kuler)
Complementary:
Hue adjacency in a theme (Kuler)
Complementary:
Data:
Hue adjacency in a theme (Kuler)
In template theory, diagonals should be uniform
Hue adjacency in a theme (Kuler)
In template theory, diagonals should be uniform
Large dark bands indicates no rotational invariance
Hue adjacency in a theme
CL
Kuler
COLOURLovers’ has less interface bias
Templates are not present
Rating
Distance to template
Rating
Distance to template
Themes near a template score worse
Rating
Distance to template
Themes near a template score worse
- “Newbie” factor
- “Too simple” factor
Rating
Distance to template
MTurk has no interface bias: much flatter
Template Conclusions
1) Templates do not model color preferences
2) Themes near a template do not score better than
those farther away
3) Not all templates are equally popular
- Simple templates preferred (see paper)
Hue Entropy: entropy of hues along the hue wheel
Hue Entropy: entropy of hues along the hue wheel
Low Entropy
Few Distinct Colors
Hue Entropy: entropy of hues along the hue wheel
Low Entropy
High Entropy
Few Distinct Colors
Many Distinct Colors
Hue Entropy: entropy of hues along the hue wheel
Rating
Hue Entropy
Hue Entropy: entropy of hues along the hue wheel
Rating
Hue Entropy
Hue Entropy: entropy of hues along the hue wheel
Rating
Hue Entropy
Hue Entropy: entropy of hues along the hue wheel
Rating
Hue Entropy
Main Analysis Results
1. Overall preference for warmer hues and cyans
Main Analysis Results
1. Overall preference for warmer hues and cyans
2. Strong preferences for certain adjacent colors
Main Analysis Results
1. Overall preference for warmer hues and cyans
2. Strong preferences for certain adjacent colors
3. Hue templates a poor model for compatibility
Main Analysis Results
1. Overall preference for warmer hues and cyans
2. Strong preferences for certain adjacent colors
3. Hue templates a poor model for compatibility
4. People prefer simpler themes (but not too simple)
Main Analysis Results
1. Overall preference for warmer hues and cyans
2. Strong preferences for certain adjacent colors
3. Hue templates a poor model for compatibility
4. People prefer simpler themes (but not too simple)
See paper for other tests
Goals
1. Analysis
Test hypotheses and compatibility models
2. Learn Models
Predict mean ratings for themes
3. New Applications
Develop new tools for choosing colors
3.63
3.63
𝑓 𝒙 =𝑦
3.63
𝑓 𝒙 =𝑦
Mean rating over all users
3.63
𝑓 𝒙 =𝑦
Features (326 total)
- Colors, sorted colors, differences, min/max, max-in,
mean/std dev, PCA features, hue probability, hue
entropy
- RGB, HSV, CIELab, Kuler color wheel
- “Kitchen Sink”
3.63
𝑓 𝒙 =𝑦
Models
- Constant baseline: mean of training targets
- SVM-R, KNN
- Lasso
- Linear regression model with L1 norm on weights
- Solutions have many zero weights: feature selection
Dataset MAE
Constant
Baseline
KNN
SVM-R
Lasso
Lasso over
Baseline
Kuler
0.572
0.533
0.531
0.521
9%
COLORLovers
0.703
0.674
0.650
0.644
8%
MTurk
0.267
0.205
0.182
0.179
33%
Dataset MAE
Constant
Baseline
KNN
SVM-R
Lasso
Lasso over
Baseline
Kuler
0.572
0.533
0.531
0.521
9%
COLORLovers
0.703
0.674
0.650
0.644
8%
MTurk
0.267
0.205
0.182
0.179
33%
Dataset MAE
Constant
Baseline
KNN
SVM-R
Lasso
Lasso over
Baseline
Kuler
0.572
0.533
0.531
0.521
9%
COLORLovers
0.703
0.674
0.650
0.644
8%
MTurk
0.267
0.205
0.182
0.179
33%
Many more ratings per theme in MTurk
Dataset MAE
Constant
Baseline
KNN
SVM-R
Lasso
Lasso over
Baseline
Kuler
0.572
0.533
0.531
0.521
9%
COLORLovers
0.703
0.674
0.650
0.644
8%
MTurk
0.267
0.205
0.182
0.179
33%
MTurk has an average std dev of 0.33
Kuler has an average std dev of 0.72
MTurk
Test Set
Lasso Rating
Human Rating
High-rated
𝑦 = 3.90, f 𝐱 = 3.41 𝑦 = 3.79, f 𝐱 = 3.50
High-rated
𝑦 = 3.90, f 𝐱 = 3.41 𝑦 = 3.79, f 𝐱 = 3.50
Low-rated
𝑦 = 1.71, f 𝐱 = 2.27 𝑦 = 1.78, f 𝐱 = 2.25
High-rated
𝑦 = 3.90, f 𝐱 = 3.41 𝑦 = 3.79, f 𝐱 = 3.50
Low-rated
𝑦 = 1.71, f 𝐱 = 2.27 𝑦 = 1.78, f 𝐱 = 2.25
High prediction error
𝑦 = 2.74, f 𝐱 = 1.78
𝑦 = 2.22, f 𝐱 = 3.16
Model Analysis
Important Lasso Features
Positive: high lightness mean & max, mean hue probability
Important Lasso Features
Positive: high lightness mean & max, mean hue probability
Negative: high lightness variance, min hue probability
Goals
1. Analysis
Test hypotheses and compatibility models
2. Learn Models
Predict mean ratings for theme
3. New Applications
Develop new tools for choosing colors
1. Improve a Theme
Maximize regression score
Stay within a distance 𝑑𝑚𝑖𝑛 of original
(L2 in CIELab)
Select order which maximizes score
Optimize colors with CMA [Hansen 1995]
Original
Best Order
Color and Order
f 𝐱 = 2.92
f 𝐱 = 3.04
f 𝐱 = 3.35
f 𝐱 = 3.00
f 𝐱 = 3.11
f 𝐱 = 3.37
f 𝐱 = 3.50
f 𝐱 = 3.50
f 𝐱 = 3.70
Original
Best Order
Color and Order
f 𝐱 = 2.92, 𝑦 = 3.04 f 𝐱 = 3.04, 𝑦 = 2.99 f 𝐱 = 3.35, 𝑦 = 3.40
f 𝐱 = 3.00, 𝑦 = 2.96 f 𝐱 = 3.11, 𝑦 = 3.21 f 𝐱 = 3.37, 𝑦 = 3.35
f 𝐱 = 3.50, 𝑦 = 3.72 f 𝐱 = 3.50, 𝑦 = 3.69 f 𝐱 = 3.70, 𝑦 = 3.82
Order and Color
MTurk A/B test
with original and
optimized themes
2. Choose 5 colors that best ‘represent’ an image
One approach: k-means clustering
One approach: k-means clustering
This ignores color compatibility
Optimize 5 colors that
1) Match the image well
2) Maximize regression score
Optimize 5 colors that
1) Match the image well
2) Maximize regression score
See paper for details
W/O Compatibility With Compatibility
Model
Model
MTurk A/B test
with and without
compatibility model
3. Given 4 colors for foreground, suggest background
Given 4 colors, choose 5th color to maximize score
Want contrast with existing colors
,
,…
Find next best color, 𝑑𝑚𝑖𝑛 away from previous choices
Model Suggestions
Random Suggestions
MTurk tests selecting
‘Worst’ and ‘Best’
4 model & 4 random
Model Limitations & Future Work
Features
Weights
Hard to interpret
Model has very few abstract colors, only 1-D spatial layout
VS
Model does not understand how colors are used
VS
Conclusions
Color preferences are subjective, but analysis reveals many
overall trends
Simple linear models can represent compatibility fairly well
Models can be useful for color selection tasks
Our datasets and learned models are available online