arXiv:1508.04785v1 [cs.CV] 19 Aug 2015
Who are the Devils Wearing Prada in New York City?
KuanTing Chen
Kezhen Chen∗
Peizhong Cong
National Taiwan University,
Taipei, Taiwan
ktchen@cmlab.csie.ntu.edu.tw
University of Rochester,
Rochester, New York, USA
kchen33@u.rochester.edu
University of Rochester,
Rochester, New York, USA
cong314159@gmail.com
Winston H. Hsu
Jiebo Luo
National Taiwan University,
Taipei, Taiwan
whsu@ntu.edu.tw
University of Rochester,
Rochester, New York, USA
jluo@cs.rochester.edu
ABSTRACT
Fashion is a perpetual topic in human social life, and the
mass has the penchant to emulate what large city residents
and celebrities wear. Undeniably, New York City is such
a bellwether large city with all kinds of fashion leadership.
Consequently, to study what the fashion trends are during
this year, it is very helpful to learn the fashion trends of
New York City. Discovering fashion trends in New York
City could boost many applications such as clothing recommendation and advertising. Does the fashion trend in
the New York Fashion Show actually influence the clothing
styles on the public? To answer this question, we design
a novel system that consists of three major components:
(1) constructing a large dataset from the New York Fashion Shows and New York street chic in order to understand
the likely clothing fashion trends in New York, (2) utilizing
a learning-based approach to discover fashion attributes as
the representative characteristics of fashion trends, and (3)
comparing the analysis results from the New York Fashion
Shows and street-chic images to verify whether the fashion
shows have actual influence on the people in New York City.
Through the preliminary experiments over a large clothing
dataset, we demonstrate the effectiveness of our proposed
system, and obtain useful insights on fashion trends and
fashion influence.
Categories and Subject Descriptors
H.4.0 [Information Systems Applications]: General; I.4.8
[Scene Analysis]: Color, Shape, Object recognition; I.5.4
[Applications]: Computer vision
Keywords
Fashion trend; clothing attributes; clothing style
∗
KuanTing Chen and Kezhen Chen contributed equally to
this work.
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than
ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission
and/or a fee. Request permissions from Permissions@acm.org.
MM’15, October 26–30, 2015, Brisbane, Australia.
c 2015 ACM. ISBN 978-1-4503-3459-4/15/10 ...$15.00.
DOI: http://dx.doi.org/10.1145/2733373.2809930.
Fashion Trend
Black
No sleeve
Pants
2014 New York Fashion Show
2014 Street-chic
(a)
Fashion Trend
White
Medium sleeve
Skirt
2015 New York Fashion Show
(b)
2015 Street-chic
Figure 1: Examples for the influence of New York
Fashion Show on the public.
1.
INTRODUCTION
”The devil wears Prada” is a very popular fashion related movie which describes the story between a recent college graduated girl and the diabolic fashion magazine editor Miranda Priestly. Although Miranda’s grim manner of
management is impressive, we also admire her unique foresight and aesthetics of fashion. In modern times, a growing number of people pay more attention to fashion and
the mass has the penchant to emulate what large city residents and celebrities wear. Moreover, investigating fashion trends has become a great interest for the industry and
academia [8][14][15][13] because of its promising opportunity
for boosting many emerging applications such as clothing
recommendation, advertising by clothing brand association,
etc. Consequently, learning the fashion trends every year
is very important for the industry as well as sociology and
psychology.
Indisputably, New York City is one of the most famous
fashion metropolises and the New York Fashion Week is one
of the four most significant fashion weeks around the world
[6][2][4]. The New York Fashion Show consists of almost
300 catwalk shows which are attended by more than 100,000
people [6]. The New York City Economic Development Corporation reported that the local economy is boosted by 850
million dollars by the New York Fashion Week [5].
A traditional way to discover fashion trend in the New
York Fashion Week would be relying on the domain knowledge of the clothing designers and experts. However, it is
very time consuming and the fashion trends vary with the
Body Region
Torso
Year #designer 2014 27 #images Year #designer #images 4000 2015 20 3914 Upper left arm
Lower left arm
Upper right arm
Lower right arm
Upper left leg
Lower left leg
Upper right leg
Lower right leg
Features Types
SIFT
Texture
Color
Skin Probability
Aggregation
Methods
Average pooling
Max pooling
(a) New York Fashion Shows
Figure 3: An illustration of the visual features we
extracted from photos.
Year #images Year #images 2014 471 2015 575 (b) Street-Chic
Figure 2: Examples from two data resources for
clothing style analysis, (a) New York Fashion Show
and (b) Street-chic.
season. Hidayati et al. [12] proposed to exploit fashion elements by clustering visual patches in the video frames of the
fashion show. In contrast to this work, we focus on learning
the clothing fashion trends by fashion attribute discovery.
Moreover, to our best knowledge, this is the first work to
address the question ”Does the fashion trend in the New
York Fashion Show actually influence the clothing styles of
the public?” (cf. Figure 1).
In this paper, we first construct a large clothing dataset
from two resources: (1) a large number of images from fashion shows during the annual New York Fashion Week. (2)
Street-chic images after New York Fashion Shows in order to
gauge whether the fashion trends have influence on people’s
daily life. Next, we analyze 2014 and 2015 fashion show images and extract the fashion attributes. More specifically,
for every image, we automatically extract 60 clothing attributes such as collar, necktie, pants, etc. Using semantic
clothing attributes to represent fashion trends can tell people the most popular style, which is not only a good way
to learn fashion trends for research, but also is a specific
fashion reference for people to choose. Finally, we utilize
these attributes as our predicted clothing fashion style and
compare these attributes with attributes from street-chic. In
our experimental results, we demonstrate the effectiveness of
the clothing attributes and the correlation of clothing style
between fashion shows and street-chic.
3.
3.1
DATASET CONSTRUCTION
In this work, we mainly exploit the fashion trend in 2015
and the influence of fashion trends on people’s daily life. In
our dataset, in order to study the fashion influence on people’s daily life, we collect two datasets. (1) Street-chic
dataset. This dataset contains 471 street-chic images in
New York from April 2014 to July 2014 and 575 street-chic
images in New York from April 2015 to July 2015 [1][3]. It
is worth noting that the number of images from street chic
is not very large in each year, but these images are high
quality and representative (cf. Figure 2) (2) New York
Fashion Show. As these street-chic images are all from
summer/spring season, we collected 3914 images from 2014
summer/spring New York Fashion Show and 4000 images
from 2015 summer/spring New York Fashion Show, respectively [7]. The 7914 images from 2014 and 2015 New York
Fashion Shows are used to extract features and fashion attributes to analyze fashion trends. We summarize and illustrate examples of our datasets in Figure 2.
Feature Extraction
Before learning the 60 clothing attributes, we need to extract clothing features beforehand to train classifiers for every attribute. Thanks for Marcin Eichner’s team [11], we apply their pose estimation software to detecting the model’s
pose and retrieve the body region of the model. More clearly,
the pose estimation process is to segment body into nine
parts: torso, upper left arm, upper right arm, lower left
arm, lower right arm, upper left leg, upper right leg, lower
left leg and lower right leg. Next, we extract four different
kinds of features (e.g. color, texture, SIFT, and skin) and
utilize two different aggregation methods to generate a visual feature vector for all the parts of the body. An example
is illustrated in Figure 3.
3.2
2.
FASHION STYLE DISCOVERY
To provide a significant and specific clothing fashion reference, an effective framework for exploring clothing style
representation is required. Using an appropriate clothing
representation, we could offer a semantic and intuitive way
for people to determine what fashion style they would like to
choose. Inspired by the paper [10], a learning-based clothing attributes approach was carried out to describe clothing style. In the research [10], Chen et al. only detected
42 upper body clothing attributes. However, we observe
the clothing information in the lower body is an essential
clue for fashion trends understanding (e.g. wearing pants or
skirt). Different from this research [10], we proposed to exploit a 60-attribute semantic representation to describe both
upper and lower body clothing attributes. In the following,
we briefly describe the adopted approaches for learning the
clothing attribute. Table 1 shows the details of clothing attributes.
Fashion Attribute Learning
In order to determine whether an fashion element is present
in an image, we adopt a Support Vector Machine (SVM) [9]
with a Chi-square kernel to learning fashion attributes (cf.
Table 1). The most intuitive way for training each fashion attribute is concatenating 72 features (cf. Section 3.1)
into a long vector that becomes the full body visual feature
vector. However, the influence of different types of feature
on each attributes may vary. For example, texture features
might have a great effect on pattern based attributes. Consequently, we compute the classification performance of each
feature to represent the importance of features as weights
towards individual attributes. As a result, a weighting parameter can be applied to vectors from different features to
emphasize the importance of different features.
3.3
Attribute Relation Inference
In section 3.2 the fashion elements are considered as isolated attributes. However, it is highly possible that some
attributes appear in pairs (or groups). For example, we ob-
Table 1: A summary of Fashion attributes.
Gender
Style
Neck9e
Placket
Upper
-­‐body Lower
-­‐body Bags
Accessories
Collar
Neckring
Pant
Skirt
Skin Exposure
Belt
Style
A"ribute 2014 2015 A"ribute 2014 2015 2014 2015 A"ribute Neck3e 0.7% 1.1% Medium 48.8% 43.4% (Knee Length) T-­‐shirt (category) 6.1% 7.7% Placket 29.6% 34.1% Long 45.3% 52.7% (Knee Length) Outwear (category) 44.6% 49.1% Skin Exposure 30% 27.4% V-­‐shape (Neckline) 15.4% 18.9% Suit (category) 2.0% 2.2% No Sleeve 35.2% 29.6% Round (Neckline) 60.0% 59.1% Tank Top (category) 28.1% 34.0% 43.6% Short Sleeve 18.5% 20.8% Other (Neckline) 19.6% 18.1% Dress (category) 43.7% 37.4% 2.3% Long Sleeve 56.6% 61.0% Shirt (category) 2.6% 3.2% 6.2% Short (Knee Length) 27.6% 22.1% Sweater (category) 3.8% 3.5% Pa?ern Color floral, graphics, plaid, solid, spot, stripe black, blue, brown, cyan, gray, green, purple, red, white, yellow, mul9color
Pa?ern Color Skirt 74.0% 71.0% floral, graphics, plaid, solid, spot, stripe black, blue, brown, cyan, gray, green, purple, red, white, yellow, mul9color
Pants 26.0% 29% Belt 21.4% 20.4% Collar 39.5% Male 2.1% Neckring 5.4% serve that a plaid shirt might have more than two colors.
Note that inter-attribute dependencies are not always symmetric. For example, while a plaid shirt strongly suggests
the presence of more than two colors, more than two colors
do not necessarily suggest a shirt being plaid. We adopt
a Conditional Random Fields (CRF) approach to inference
the relation betweens attributes. More specifically, each attribute acts as a node in the CRF framework and the edge
connecting every two nodes indicates the joint probability of
these two attributes. We build a fully connected CRF with
all the attributes pairwise connected.
4.
Table 2: The summary of attributes percentage appeared in spring/summer fashion shows.
EXPERIMENTAL RESULTS
We conduct experiments on a large clothing dataset collected from the New York Fashion Shows and Street-chic
photos.
We conduct several experiments to gain understanding of
the performance of the attributes. The overall accuracy of
attributes is 62.6%. The examples in Figure 1 indicate that
the extracted attributes was effective for clothing style representation. An interesting observation is that some categories suffer from worse results (e.g. 42% accuracy of belt,
56% accuracy of accessories) since the objects are relatively
small compared to the entire body. In the future, we could
segment the body into more parts to improve the accuracy of
the attributes detection. For example, we could segment the
middle body for belt and the neck part for the accessories.
Table 2 shows the attributes percentage detected on the
2014 New York Fashion Shows and 2015 New York Fashion Shows. We divide these attributes into three categories,
color, pattern and clothing style. We observe that there
are always some classic colors, patterns or styles that have a
large amount of clothing images in the fashion shows of every
year. For example, in upper body color, both 2014 and 2015
fashion images have a large amount of white, gray and black
colors. In upper body and lower body patterns, solid pattern
is the classic pattern and solid pattern clothes always dominate every year’s fashion shows. In the style attribute category, there are many images with skirt in spring/summer
fashion shows. These examples indicate that the results generated by attribute description methods are very reasonable
and these results can be a very significant reflection for the
fashion shows styles.
Besides these classic cloth styles, we are also interested
in the changes in the fashion trend. These changes in attributes could indicate special fashion trends for the specific
year. Note that we remove the classic style attributes, black,
white and gray in the color category, solid pattern in the pattern category and skirts in the style category to present the
changes more clearly. Figure 4 shows the change of fashion
trend between the 2015 fashion shows and the 2014 fashion
shows. From this figure, we could tell the unique fashion
trends for 2015. In the color category, attributes with larger
changes in the upper body are blue, multicolor, cyan and
Bags 21.9% 22.7% accessories A"ribute 2014 2015 Color
A"ribute A"ribute A"ribute A"ribute 2014 2015 2014 2015 2014 2015 2014 2015 (Upper-­‐body) (Upper-­‐body) (Lower-­‐body) (Lower-­‐body) Black 20.9% 19.2% Blue 8.9% 11.7% Purple 3.2% 3.1% Black 27.7% 28.5% Red 4.1% 5.4% Blue 5.7% 6.8% Purple 0.8% 1.1% Red 9.0% 8.9% Brown 4.0% 4.4% White 32.6% 32.1% Brown 6.5% 5.5% White 29.9% 29.3% Cyan 10.4% 11.8% Yellow 14.6% 11.7% Cyan 0.3% 0.5% Yellow 6.3% 4.3% Gray 18.9% 18.4% Mul3color 14.2% 16.3% Gray 15.7% 15.7% Mul3color 36.4% 35.5% Green 7.1% Green 6.9% 2.1% 1.6% Pattern
A"ribute A"ribute A"ribute A"ribute 2014 2015 2014 2015 2014 2015 2014 2015 (Upper-­‐body) (Upper-­‐body) (Lower-­‐body) (Lower-­‐body) Floral 12.7% 10.0% Solid Graphics 14.7% 13.7% SpoYed 3.1% 3.2% Graphics 0.5% Striped 2.6% 3.3% Plaid Plaid 0.3% 46.4% 50.4% Floral 8.9% 8.7% Solid 59.1% 62.4% 20.0% 19.5% SpoYed 1.7% 1.2% 0.6% Striped 1.5% 2.1% 0.5% red; attributes with larger changes in the lower body are
blue and purple. In the pattern category, attribute with
larger changes in the upper body is striped; attribute with
larger changes in the lower body is striped. In the style category, long knee length, tank top, placket, outwear and collar
are most significant attributes in 2015. These attributes
with larger changes could be regarded as the unique fashion
attributes trends shown in 2015.
In order to analyze the influence from the New York Fashion Shows on people’s daily life, we analyze the clothing attributes from the street chic images. As we mainly focus
on the spring and summer fashion show influence, we collected 471 street chic images in 2014 and 575 street chic
images in 2015 that were taken in New York from April
to July and utilize these images as our street-chic data for
fashion influence analysis. The changes of these attributes
between 2014 and 2015 are shown in Figure 4. As shown in
Figure 4, some changing trends of attributes are the same
in fashion shows and in street chic. For example, there is
a significant rise in both placket and tank top of the style
category. In the pattern category, striped pattern in the
upper body increased substantially. In the color category,
blue in the lower body, blue, cyan, red and multicolor in the
upper body showed an upward trend in both fashion show
and street chic images. Consequently, we could tell that
many people have the penchant to emulate clothing styles
shown in the fashion shows. Some examples are shown in
Figure 5. Interestingly, there are some different changes of
fashion elements between the fashion shows and street chic
images. For instance, we observe that the difference between
the fashion shows and street chic lies in the long knee sleeve
and outwear. We discovered that summer 2014 is the coldest in a decade, and people tended to wear outwear to keep
warm. Another interesting observation is that people tend
to wear bright-colored clothes in both the upper body and
the lower body instead of wearing in only the upper body as
shown in the fashion shows. A reasonable explanation is that
light-colored clothes can absorb less sun light and keep people cooler; therefore, people are tempted to wear light color
clothes in order to keep them cooler in the hotter weather in
2015. In summary, it seems clear that fashion trends from
fashion shows indeed provide people a clothing reference and
have significant influence on people’s daily life. However, so-
-­‐4.0% -­‐5.0% Suit (Category) Dress (Category) Outwear (Category) Tank Top (Category) Shirt (Category) T-­‐shirt (Category) Long (kneelength) Sweater (Category) Short (kneelength) Medium (kneelength) Long (sleeve) Other (neckline) Lower_stripe Lower_spot Lower_plaid Lower_graphics 4.0% 3.0% 2.0% 1.0% -­‐4.0% Upper_red Upper_purple Upper_yellow Upper_green Upper_mulCcolor Upper_cyan Upper_brown Lower_red Upper_blue Lower_yellow Lower_purple Lower_green Lower_mulCcolor -­‐3.0% Lower_blue -­‐2.0% Lower_cyan 0.0% -­‐1.0% Lower_brown Upper_red A.butes Upper_yellow Upper_purple Upper_mulCcolor Upper_cyan Upper_green Upper_brown Lower_red Upper_blue Lower_yellow Lower_purple Lower_mulCcolor Lower_green Lower_blue Lower_cyan 0.0% The percentage of a(ribute 1.0% Lower_brown The percentage of a(ribute A.butes 2.0% -­‐4.0% Lower_floral -­‐4.0% Street Color Change Color Change 3.0% -­‐3.0% Upper_striped Upper_spo:ed Upper_floral -­‐3.0% Upper_plaid 0.0% -­‐2.0% -­‐5.0% 4.0% -­‐2.0% V-­‐shape (neckline) 1.0% -­‐1.0% A.butes -­‐1.0% Round-­‐Shape (neckline) Placket No (sleeve) 2.0% Upper_graphics The percentage of a.bute Lower_stripe Lower_spot Lower_plaid Lower_graphics Lower_floral Upper_striped Upper_spo:ed Upper_floral Upper_plaid 0.0% Upper_graphics Short (sleeve) Belt Collar A.butes Street Pa(ern Change 3.0% -­‐3.0% NeckCe 0.0% -­‐4.0% Neckring Suit (Category) Dress (Category) Tank Top (Category) Shirt (Category) T-­‐shirt (Category) Outwear (Category) Long (kneelength) Sweater (Category) Short (kneelength) Medium (kneelength) Long (sleeve) Other (neckline) 2.0% -­‐8.0% 1.0% -­‐2.0% 4.0% -­‐2.0% Pa(ern Change 2.0% -­‐1.0% 6.0% -­‐6.0% A.butes 3.0% The percentage of a.bute V-­‐shape (neckline) Placket No (sleeve) Short (sleeve) Belt Collar NeckCe -­‐8.0% Round-­‐Shape (neckline) -­‐6.0% Skin Exposure -­‐4.0% Neckring 0.0% 8.0% bags Accesssories 2.0% The percentage of a(ribute 4.0% bags Accesssories The percentage of a(ribute 6.0% -­‐2.0% Street Style Change 10.0% 8.0% Skin Exposure Style Change 10.0% A.butes Figure 4: Quantitative fashion style changes in 2015
shown in the New York Fashion Shows (left column)
and street-chic (right column). The correlation coefficients between the two columns are (style: -0.32,
pattern: 0.64, color: 0.28). Note that the positively
correlated attributes are marked red and the negatively correlated attributes are marked green. In
terms of style, there are positively correlated attributes even though the overall correlation is negative, revealing both correlation and distance between stage and reality.
cial condition and natural conditions, such as weather, could
be important factors for people to determine what they are
likely to wear. Furthermore, analysis of street chic images
also has benefit to inspire fashion designers to come up with
more main-stream designs.
5.
CONCLUSIONS
In this work, we construct a large dataset from the New
York Fashion Shows and street-chic in order to investigate
the possible clothing fashion trends in New York. In addition, we investigate a learning based method to automatically detect fashion attributes as the representation of fashion trends. We further demonstrate that the proposed framework is effective in fashion trend discovery. Furthermore,
we compare the analysis result from the New York Fashion
Shows and street-chic images in order to investigate whether
and to what extent the fashion shows have influence on New
York people. In the future, we plan to collect more images to increase the data set for more comprehensive studies.
Moreover, there is also keen interest in exploring the proper
fashion outfits to recommend users by aggregating user preference in clothing style and one future direction would be to
incorporate these elements into the existing model and the
scope will be extending and applying our system to practical
applications, for example clothing style and product recommendation. Besides, the concept of clustering will also be
one of the options as it will be interesting to automatically
learning unseen clothing elements [12].
6.
ACKNOWLEDGMENTS
This work was generously supported in part by Google,
Yahoo, Adobe, TCL, and New York State CoE CEIS and
IDS. KuanTing Chen is supported by a scholarship by the
Figure 5: The examples of the similarity of fashion
trends shown both in (a) New York Fashion Show
and (b) Street-chic.
Taiwan Government NSC Study Abroad Program grants
103-2917-I-002-159.
7.
REFERENCES
[1] Pinterest. http://pinterest.com.
[2] In vogue: How does catwalk influence the main street.
http://www.bbc.com/news/magazine-14984468.
[3] Flickr. http://www.flickr.com.
[4] The 14 best trends of new york fashion week.
http://www.marieclaire.com/fashion/g2359/new-yorkfashion-week-spring-2015-trends/.
[5] The new york city economic development corporation.
http://www.nycedc.com/.
[6] The new york times. http://www.nytimes.com/.
[7] Vogue. http://www.vogue.com/.
[8] L. D. Bourdev, S. Maji, and J. Malik. Describing
people: A poselet-based approach to attribute
classification. In IEEE ICCV 2011, Barcelona, Spain,
November 6-13, 2011, pages 1543–1550, 2011.
[9] C.-C. Chang and C.-J. Lin. Libsvm: A library for
support vector machines. ACM Trans. Intell. Syst.
Technol., 2(3):27:1–27:27, May 2011.
[10] H. Chen, A. Gallagher, and B. Girod. Describing
clothing by semantic attributes. In ECCV 2012,
volume 7574, pages 609–623, 2012.
[11] M. Eichner and V. Ferrari. Appearance sharing for
collective human pose estimation. In Computer Vision
- ACCV 2012, pages 138–151, 2012.
[12] S. C. Hidayati, K.-L. Hua, W.-H. Cheng, and S.-W.
Sun. What are the fashion trends in new york? In
Proceedings of the ACM International Conference on
Multimedia, MM ’14, pages 197–200. ACM, 2014.
[13] S. Liu, J. Feng, C. Domokos, H. Xu, J. Huang, Z. Hu,
and S. Yan. Fashion parsing with weak color-category
labels. IEEE Transactions on Multimedia,
16(1):253–265, 2014.
[14] S. Liu, Z. Song, G. Liu, C. Xu, H. Lu, and S. Yan.
Street-to-shop: Cross-scenario clothing retrieval via
parts alignment and auxiliary set. In IEEE CVPR
2012, pages 3330–3337, 2012.
[15] T. V. Nguyen, S. Liu, B. Ni, J. Tan, Y. Rui, and
S. Yan. Sense beauty via face, dressing, and/or voice.
In Proceedings of the 20th ACM Multimedia
Conference, MM ’12, pages 239–248, 2012.