Discovering Pictorial Brand
Associations from Large-Scale
Online Image Data
Gunhee Kim*
Eric P. Xing
School of Computer Science,
Carnegie Mellon University
*: (now with Disney Research Pittsburgh)
December 7, 2013
1
Outline
• Problem Statement
• Dataset
• Algorithm
• Visualization – Brand Association Maps
• Results
• Conclusion
2
Brand Equity
A set of values or assets linked to a brand's name and symbol
• Bearing competitive advantage over its competitors
• Boosting efficiency and effectiveness of marketing programs
• Attaining price premium due to increased customer satisfaction and loyalty
Interbrand
(2013)
3
Brand Equity
A set of values or assets linked to a brand's name and symbol
• Bearing competitive advantage over its competitors
• Boosting efficiency and effectiveness of marketing programs
• Attaining price premium due to increased customer satisfaction and loyalty
• [Keller 1993]
Brand Knowledge
Brand Awareness
Brand
recall
Brand
Recognition
• [Aaker 1991]
Brand
loyalty
Name
awareness
Brand Image
Types of
brand
association
Favorability of
brand
association
Strength of
brand
association
Uniqueness of
brand
association
Brand Equity
Perceived
quality
Brand
association
Other properties
(ex. trademarks, patens)
4
Brand Equity
A set of values or assets linked to a brand's name and symbol
• Bearing competitive advantage over its competitors
• Boosting efficiency and effectiveness of marketing programs
• Attaining price premium due to increased customer satisfaction and loyalty
• [Keller 1993]
Brand Knowledge
Brand Awareness
Brand
recall
Brand
Recognition
• [Aaker 1991]
Brand
loyalty
Name
awareness
Brand Image
Types of
brand
association
Favorability of
brand
association
Strength of
brand
association
Uniqueness of
brand
association
Brand Equity
Perceived
quality
Brand
association
Other properties
(ex. trademarks, patens)
5
Brand Associations
(What Comes to Mind When You Think of …)
A set of associations that consumers perceive with the brand
• Top-of-mind attitudes or feelings toward the brand
basketball, golf, …
soccer, national teams, …
swimming, diving, beach, …
6
Brand Associations
(What Comes to Mind When You Think of …)
A set of associations that consumers perceive with the brand
• Top-of-mind attitudes or feelings toward the brand
• Consumer-driven brand equity
I associate
with
I need a men’s golf shirts… What would I buy?
How can we find out people’s brand associations?
7
A Traditional Way to Brand Associations
Collect direct consumer responses to carefully-designed questionnaires
In-N-Out [Dane et al. 2010]
Reese’s [Till et al. 2011]
• Expensive and time-consuming
Cheap and immediate
• Sampling bias /common methods bias
Candid impression from a large
crowd of potential customers
Can Web data save us?
8
‘ ’s Brand Association Map
The most important concepts and themes that consumers have
• Built from consumer-generated text data on online social media
(eg. Weblogs, Boards, Wiki)
9
Our Idea: Photo-based Brand Associations
Take advantage of large-scale online photo collections
• No previous attempts so far to leverage the pictures
Our underlying rational
• Take a picture and tag as burger+king
• Pictorial opinion on burger+king
• Crawl millions of online images
tagged as burger+king
• We can read people’s candid pictorial
opinions on burger+king
Pictures cannot replace texts. They are complementary.
ex) Nike shoes are too expensive. Their design is tacky!
10
Our Idea: Photo-based Brand Associations
Take advantage of large-scale online photo collections
• No previous attempts so far to leverage the pictures
Why are online images helpful to discover brand associations?
1. Brands are recognized by images.
2. Take or bookmark pictures of the
products you wish to buy.
11
Our Idea: Photo-based Brand Associations
Take advantage of large-scale online photo collections
• No previous attempts so far to leverage the pictures
Why are online images helpful to discover brand associations?
3. Capture interactions between users and products
in natural social context
12
Objective of This Paper
Two visualization tasks as a first technical step
1. (Image-level) Detect and visualize key pictorial concepts
associated with brands
• Identifying a small number of exemplars / image clusters
• Project the clusters into a circular layout
13
Objective of This Paper
Two visualization tasks as a first technical step
2. (Subimage-level) Localize the regions of brand in images
• A basic function to reveal interactions between users and products
• Potential benefits: content-based retrieval and online multimedia ad
14
Outline
• Problem Statement
• Dataset
• Algorithm
• Visualization – Brand Association Maps
• Results
• Conclusion
15
Image Dataset from Five Websites
4,720,724 images of 48 brands of four categories
Luxury
Sports
Beer
Fastfood
Two largest and most popular
photo sharing Web communities
Various forms of artwork by users
Pictures bookmarked by users
Photo hosted for twitter
16
Outline
• Problem Statement
• Dataset
• Algorithm
• Visualization – Brand Association Maps
• Results
• Conclusion
17
Approach – KNN Graph Generation
Input: a set of images of a brand (e.g. louis+vuitton):
I = {I1,..., I N }
Feature extraction
• Dense feature extraction of Color SIFT and HOG
• Histogram intersection for similarity measure
KNN Similarity graph
G = (I, E)
• Approximate method (Multiple random divide-and-conquer using meta-data)
[Wang et al. 2012] in O(NlogN)
18
Approach – Exemplar Detection/Clustering
Detecting L number of exemplars (A Ì I)
• A small set of representative images
• Diversity ranking algorithm (temperature maximization) [Kim & Xing 2011]
• Solving submodular optimization on
G = (I, E)
19
Approach – Exemplar Detection/Clustering
l L
Clustering ({C }l=1 )
• Each image I is associated with its closest exemplar
• Random walk model (What exemplar is most likely reachable from each I?)
20
Approach – Brand Localization via
Cosegmentation
Find the regions that are most relevant to the brand
• Use MFC algorithm [Kim&Xing. 2012] to each cluster of coherent images
• Make parallelization easy and prevent cosegmenting images of no
commonality
• Optionally, generate bounding boxes
21
Approach – Closing a Loop
Go back to graph generation
(Our argument) even imperfect segmentation helps obtain
better image similarity measurement.
• Not robust
against the
location and
scale change
• Image similarity
as the mean of
best matched
segment similarity
22
Outline
• Problem Statement
• Dataset
• Algorithm
• Visualization – Brand Association Maps
• Results
• Conclusion
23
Embedding on a Circular Layout
Goal: Compute two coordinates (r, q ) of key clusters
• Radial distance: a larger cluster closer to the center
• Angular distance: the smaller, the higher correlation
(Based on
Nielson’s BAM)
Radial distance
• Compute stationary distribution of nodes
p = PT p where p 1 =1, P = l P + (1- l )vT
• Sum of stationary distributions of nodes
of each cluster  the probability that
a random walker stays
Angular distance
Radial distances
24
Embedding on a Circular Layout
Goal: Compute two coordinates (r, q ) of key clusters
• Radial distance: a larger cluster closer to the center
• Angular distance: the smaller, the higher correlation
(Based on
Nielson’s BAM)
Angular distance
• Pairwise similarity btw clusters S using the
random walk with restart [Sun et al. 2005]
• Using spherical Laplacian eigenmap
[Carter et al. 2009.]
Angular distance
q = argmin ååS(a, b) q a - q b - åå q a - q b
a
b
a
g
b
Similar clusters should have Penalty to avoid
a trivial solution
smaller angle difference
Radial distances
25
Outline
• Problem Statement
• Dataset
• Algorithm
• Visualization – Brand Association Maps
• Results
 Examples of Brand Association Maps
 Quantitative results for exemplar detection/clustering
 Quantitative results for brand localization
 Correlations btw Web image data and actual sales data
• Conclusion
26
Experiments – Brand Association Maps
Weddings
Brands' characteristic
visual themes
Events sponsored
by brands
27
Experiments – Exemplar Detection/Clustering
Groundtruth for clustering accuracy
• Randomly select 100 sets of three (i,j,k) images per class
• Manually select which of (j,k) is closer to i :
(j
k | i)
• Accuracy is measured by how many sets are correctly clustered
Similarity btw the clusters of image i and j
Wilcoxon-MannACC :=
Whitney statistics
å
(i, j,k )
I( j
k | i Ù s (C j , Ci ) > s (Ck ,Ci ))
å
(i, j,k )
I( j
k | i)
100
Sub-M: Multiple runs of our clustering + cosegmentation
Sub: Our clustering without cosegmentation
Kmean/Spect: K-mean clustering / Spectral clustering
28
LP: Label propagation [Raghavan et al. 2007] AP: Affinity propagation [Frey & Dueck 2008]
Experiments – Brand Localization
Task: Foreground detection
• Manually annotate 50 images per class
• Accuracy is measured by intersection-over-union Acc =
GTi Ç Ri
GTi È Ri
MFC-M: Multiple runs of our clustering + cosegmentation
MFC-S: Single run of our clustering + cosegmentation
MFC: Random partition + our cosegmentation
COS: Submodular optimization [Kim et al. ICCV11]
LDA: LDA-based localization [Russell et al. CVPR06]
29
Experiments – Brand Localization
Task: Foreground detection
• Manually annotate 50 images per class
• Accuracy is measured by intersection-over-union Acc =
GTi Ç Ri
GTi È Ri
Examples
30
Experiments – Correlation with Sales Data
Photo volumes vs. Market share
• Nike’s market share is 57.6% in sports brands. How’s about image volumes?
• Based on brands’ annual reports
Observations
• Ranking are roughly similar, but the proportions do not agree.
• Guinness : premium beer (more photo data)
Taco+bell : cheap fastfood brand (fewer photo data)
31
Outline
• Problem Statement
• Dataset
• Algorithm
• Visualization – Brand Association Maps
• Results
• Conclusion
32
Conclusion
Study of brand associations from millions of Web images
• Complementary views on the
brand associations beyond texts
• Experiments on 5 Millions images of
48 brands from five popular websites
Jointly achieving two levels of visualization tasks
• Visualizing core pictorial concepts associated with brands
• Localizing the regions of brand in images
Various potential applications
• Online multimedia contextual advertisement, competitor mining
33
Thank You!
Extended version will be presented in
Press Release
http://www.cs.cmu.edu/news/carnegie-mellon-researchers-createbrand-associations-mining-millions-images-social-media-sites
34