Harmonic Colorization Using Proportion Contrast
Catherine Sauvaget ∗
University of Paris 8
2 rue de la Liberté,93526 SAINT-DENIS cedex
Saint-Denis, FRANCE
Vincent Boyer �
University of Paris 8
2 rue de la Liberté,93526 SAINT-DENIS cedex
Saint-Denis, FRANCE
Figure 1: left: original image; center: harmonization on red and green colours; right: harmonization on orange, red and green colour.
Abstract
We present a proportion colour harmony model enhancing image
aesthetics in terms of human colour perception. Given an arbitrary
image, colours are modified to improve their proportion harmony.
The proportion relationship between colours has been described in
theories of colours but never proposed in computer graphics. Our
model detects proportion harmony or the lack thereof harmony, proposes improvements verifying constraints defined by the user as
well as different possibilities to recolor the image by shifting hues.
Our technique is well-adapted to adjust the colour in an image, in
a part of an image, image composition or a subset of colours depending on the user choices. Results show that our colour harmonization model allows the user to handle classical images such as
photographs or drawings and paintings.
CR Categories:
I.4.3 [Computing Methodologies]:
Processing—Enhancement
Image
Keywords: Colour harmonization, Non-Photorealistic techniques
1
Introduction
Colour is one of the major information vectors of an image. Image
perception necessarily depends on the colours used and the relationship between these colours. Harmonic colours tend to provide a
pleasant visual perception. Artists know these colour relationships
and use them to create effects. Generally they use their own colour
∗ e-mail:
cath@ai.univ-paris8.fr
� e-mail: boyer@ai.univ-paris8.fr
palette and their arrangements may produce intense psychological
effects. Even if colour combinations are perceived differently, rules
of colour harmonies exist and are widely used by artists. Other
users, not aware of colour theories, also prefer harmonized images
depending on the context or their cultural background.
Existing tools and research mainly focus on colour relationship
without taking into account the proportion of each colour in an image. A pleasant visual perception defining harmony is not only due
to the colour choices in an image but depends also on the proportion
of each colour.
We propose a new colour harmonization model based on colour
proportions. It uses the contrast of extension concept proposed by
Johannes Itten [Itten 1961]. The user can define some constraints
(hue values, proportion) and our system detects the harmony or lack
of harmony, proposes numerical solutions in terms of pixel reassignment and recolour the image according to the user’s choice.
Our application can deal with photographs or stylized images. It
can be applied to an image, only a part of it, a set of colours or to
an image composition process.
In this paper, after a presentation of the main theories of colour harmony rules, we present our model to recolorize images preserving
a proportion colour harmony. We detail how to detect colour harmony, how to find a proportion solution for colour harmonization
if needed and how to change the hue of pixels to create the desired colour harmony. A lot of examples are given and commented,
demonstrating the usefulness of our model.
2 Previous work
In both philosophical and scientific domains, the notion of ’colour
harmony’ drew considerable interest. Since the early theories produced by Goethe [von. Goethe 1810], Chevreul [Chevreul 1839],
Young and Maxwell [Maxwell 1860], colour harmony deals with
the distinction between harmonious and non-harmonious combinations of colours. Generally a terminology is created to depict complementary (or non-complementary) relationship between different
colours. As an example, “médiocre” or “désagréable” are used by
Chevreul to mean non or almost non-harmonic colours; “balanced”
and “unbalanced” are used by Munsell. Chevreul developed more
systematic laws of harmony, particularly laws of contrast and analogous colours. He noted that the difference between two colours is
intensified by their juxtaposition. This work influenced art movements such as impressionism and neo-impressionism. Last century,
colour harmony has also been studied by introducing new representations of colours [Munsell 1921; Ostwald and Birren 1969; Itten
1961]. Colour harmony is then described using the representation
of colours and rules to depict ’valid’ combinations. In those representations, most ’valid’ combinations correspond to colour position relationships. Other ’valid’ combinations using a quantitative
representation of colour harmony are introduced by Granville and
Jacobson [Granville and Jacobson 1944].
Itten used a colour representation close to artistic principles [Itten
1961]. A colour is mainly described by its hue and the colour system is represented by a colour wheel. Colour harmony (i.e colour
relationships) can be described through a relative position of hues
on the colour wheel. A quantitative representation of colour harmony is described and based on proportion. Itten identified seven
contrasts of colour: hue, light and dark, cool and warm, complementary, simultaneous contrast, saturation and extension (also
named proportion). For example: a two-complementary colour
contrast is represented by opposed colours on the colour wheel; a
three-complementary colour contrast is formed by an equilateral or
an isoscele triangle; a square describes a four-complementary contrast... As described by Atencia et al. [Atencia et al. 2008] these
schemes have been widely inspired by artists and more precisely by
impressionist painters and are nowadays systematically adopted by
painters and designers.
Most of existing tools [Nack et al. 2003; Color Schemer 2000;
Adobe Kuler 2006] are based on the colour harmony theories presented above. According to the requirements, they provide users
with a harmonic colour set. It is worth noticing that these tools
produce a set of colours and never harmonize a given image.
Ongoing research on colour harmonization addresses different issues from the main theories of colour harmony as in [Westland
et al. 2007] or the study of psychological assessment of colour
harmony with colour combination pairs on a grey background [LiChen and Ronnier Luo 2006]. Unfortunately, their studies are restricted to pairs of colours whereas images are generally composed
of more than two colours.
Other work on image colour harmony provides a model for the harmonization of colours preserving most parts of the original colours
of an input image [Cohen-Or et al. 2006]. This model uses predefined harmonic templates composed at most of two sectors. After
finding the one that best fits the image, this tool allows the user to
change the orientation of the hue template to globally change the
colours of the image. A graph-cut optimization is used to enforce a
continuous modification of colours in the image space. Note that an
extension to the colour harmonization for videos has been proposed
by [Sawant and Mitra 2008]. Moreover a cut-and-paste system is
described that allows the user to realize colour harmonization and
to compose scenarios. For example, colour harmonization can be
realized independently for the background and foreground and a final image can be created by composition of the latter two. This
method presents limitations: the harmonic templates are composed
only by one or two sectors and proportion colour harmony cannot
be produced; the segmentation of the image should be improved in
order to recolour the image more efficiently.
Other problems remain: using their colour schemes, the authors observe that the above method does not change natural images since
their colours usually fall almost perfectly into some harmonic template; “colour harmonization is most useful when saturated, manmade colours are present in the image” [Cohen-Or et al. 2006].
These assertions are partially in contradiction with the historical
practices that inspired Itten. As described above, artists and par-
ticularly impressionist painters use vibrant harmonic colours. A
natural image mostly composed of desaturated colours is wrongly
considered as an harmonic colour image. Moreover other artists
such as comics creators use a dominant hue to depict an atmosphere [Sauvaget and Boyer 2008] and with their model a pretreatment should be used to preserve atmospheric colour during the
colour harmonization.
We improve significantly the harmonic schemes presented above by
introducing the proportion colour harmony described by Itten. Our
model is a complementary model to already existing work. It is an
important harmony since the artists choose their colours but also the
quantity of it. This is the first implementation of such a harmony.
Our method harmonizes automatically a given image through its
colour palette and allows the user to realize composition scenarios.
3 A new harmonic model
Our notion of colour harmony is based on Itten’s colour harmony
principles. Itten defined harmony as a simultaneous judgment on
two or many colours. Note that atmosphere can be produced in
an image using two predominant colours. Following Itten, global
perception of an image is necessary to perceive its contrast. All the
colours used in an image are important, the designer uses them to
convey particular emotions to the viewer.
A hue, saturation, value model is used to describe each colour and
each colour can be placed on Itten’s colour wheel. Based on this
model, Itten defined seven contrasts: hue, saturation, light and dark,
cool and warm, complementary, simultaneous and extension (also
named proportion).
The first three refer to contrasts in the three perceptual dimensions
of colour and the last four concern contrasts in size for different
colours [Westland et al. 2007]. As mentioned above, the harmonic
schemes previously proposed describe a position relationship on
the colour wheel but the proportion relationship is never studied
and cannot be represented.
We present our proportion colour relationship. Our method allows
the user to produce colour harmony in an image or in an image
area defined by the user. The term of image is used for both in the
following subsections.
3.1
Colour proportions
Contrast of extension, sometimes named proportion contrast, focuses on the relative areas of two or more colour patches. The
colour harmony is obtained when there is a balance between the
amount of pixels of two colours. Proportion contrast is used to refer
to the contrast between two colour proportions. Colour proportion
harmonies can be obtained for 2 to 6 colours (yellow, orange, red,
violet, blue and green). Itten divided the colour wheel into six sectors to represent six different colours where hue is used for classification. As the classification depends on the hue value, completely
desaturated pixels in an image, and by extension gray-scale images,
are considered harmonious. Moreover, Itten considered that an image is harmonious if the average of its colours is a grey level.
Goethe noted that different hues produce different light values. Itten defines proportions in terms of the reciprocal of those Goethe
values. Goethe’s lighting colour values are transformed into colours
with harmonious dimensions. For example, Goethe estimated that
yellow is three times lighter than violet; Itten mentioned that to
obtain proportion colour harmony, the ratio between yellow and violet areas is 1:3. Table 1 summarizes the set of values describing
the area proportion for each colour.
yellow
3
orange
4
red
6
violet
9
blue
8
green
6
Table 1: Harmonic area proportions.
that case, CS = �O� R). This provides an easy way to preserve
particular colours or an atmosphere in a given image. Note that,
compared to the Cohen-Or et al. harmonic scheme [Cohen-Or et al.
2006], our model is not bounded to two sectors at most and permits
more possible configurations.
Considering the 360 degrees chromatic wheel, the proportional ar75◦
eas for harmonic pictures are in degrees (see figure 2): red: 60,
40◦
orange: 40, yellow: 30, green: 60, blue: 80 and violet: 90.
150◦
150◦
G Y
All proportion relationships between two to six colours can be deO
10◦
duced from this table. For example: to obtain harmonic colour proR
B
B
portion between yellow, blue and green, the amounts for the yellow
8
6
3
340◦
V
, 3+8+6
for the blue and 3+8+6
for the green.
area should be 3+8+6
250◦
250◦
75◦
70◦
◦
40
Y
150◦
G
10◦
R
340◦
V
300◦
B
40◦
Y
20◦◦
O
10
R
V
250◦
Figure 3: Examples of sector configurations. From left to
right : default configuration with 6 sectors: Y = �40� 75� 3),
O = �10� 40� 4), R = �340� 10� 6), V = �250� 340� 9), B =
�150� 250� 8), G = �75� 150� 6); 4 sectors: Y = �40� 70� 3),
R = �340� 10� 6), V = �250� 300� 9), B = �150� 250� 8), and
an incorrect configuration: Y = �40� 75� 3), O = �10� 40� 4),
R = �340� 20� 6), V = �250� 340� 9), B = �150� 250� 8), G =
�75� 150� 6).
Figure 2: Itten’s proportion colour wheel
According to Itten’s proportion colour wheel, we define six sectors on the hue wheel (see left of figure 3). A sector S is then
described by two angles Sα and Sβ and a proportion value Sp
which denoted S = �Sα � Sβ � Sp ). We always assume that the
angle between Sα and Sβ is counterclockwise. The default angles
of each sector have been deduced from the statistical mean of values given by a set of users. The proportion value is deduced from
table 1: Y = �40� 75� 3), O = �10� 40� 4), R = �340� 10� 6),
V = �250� 340� 9), B = �150� 250� 8), G = �75� 150� 6). In our
harmonic scheme, sectors are not necessary contiguous but the intersection of all sectors must be empty.
The user can redefine the number of defined sectors from 2 to 6, the
sectors angles, the proportion values and can define the number of
sectors thereof to be considered for the computation of harmony. In
the following, DS represents the list of defined colour sectors and
S represents one of the six possible colour sectors (Y , O, R, V , B,
G). A list of considered colour sectors CS is then created. Note
that CS is a subset of DS and must be at most equal to the list of
defined colour sectors DS.
If the user changes the proportion values or if he changes the order of the colour sectors, he creates new proportion considerations
and we do not ensure that the resulting image will be harmonic as
defined in different colour harmony theories.
Our system allows the user to create DS respecting the sector
intersection constraint. Figure 3 presents two different possible
sector configurations and one impossible configuration. For the
first one, we use the default values described above and DS =
�Y� O� R� V� B� G). For the second one, only 4 sectors have been
defined DS = �Y� R� V� B). Note that Yβ is no more the default
value and sectors V and R are no more contiguous. The last one
illustrates an incorrect configuration since R ∩ O �= ∅ (i.e angle
Rα Oβ is counterclockwise and Rβ Oα is clockwise).
Colour harmony is computed only on the considered sectors: the list
CS. In the first example of figure 3, the user can choose to consider
only orange and red colours in DS to compute the harmony. In
As mentioned above, atmospheres can be very important in an image. As an example, comics creators produce dramatic atmospheres
with browns and dark blues, mysterious atmospheres with dark
blues and greens, violent atmospheres with oranges and reds or
threatening atmospheres with ochres and browns. With our model,
to handle an image with a mysterious atmosphere, it is necessary to
treat sectors B and G (adapted respectively to dark blue and green)
independently of the other sectors. So colour harmony should be
achieved with DS, CS = �B� G) and/or CS = �Y� O� R� V ) independently.
Three main questions have to be solved for colour harmonization:
how to detect colour harmony, how to find a proportion solution for
colour harmonization if needed and how to change the hue of pixels
to create a colour harmony. The following subsections handle these
questions.
3.2
Proportion harmony detection
Our method allows the user to detect colour harmony or its lack in
an image.
Considering an image and the sectors given by the user, our model
assigns each pixel to a sector according to its hue value. A list
of pixels is then created per sector Ls = �Ns ; P0 � . . . � PN−1 }
with Ns = card�Ls ) and Pi �xi � yi � hi � si � vi ) a pixel of the image where �x� y) denotes the position and �h� s� v) its colour in the
HSV model. In the following Ls represents one of the six possible
lists (Ly for yellow, Lo for orange, Lr for red, Lv for violet, Lb for
blue, Lg for green). Each list is then sorted by hue and we have for
each sector:
Sα ≤ h0 ≤ . . . ≤ hN−1 ≤ Sβ
Since we obtain the number of pixels per sector, we can easily verify if the proportion colour harmony is preserved. It consists in
comparing if the number of pixels of each considered sector divided by the number of pixels to be considered, corresponds to proportions described in table 1:
340◦
∀Si ∈ CS� Ns� −
3.3
�
�card��S)
j=1
card��S)
j=1
S p� × �
Nsj
S pj
�
=0
(1)
Finding a proportion solution
Since in most cases, for a given image and a list CS chosen by the
user, the proportion colour harmony, i.e. 1, is not verified it should
be computed by our application. Our system produces a solution
that verifies the conditions presented above by assigning a new sector to some pixels. This is a classical assignment problem. To obtain a solution, we use the arc-length distance on the hue wheel
between sectors as a cost function.
For each sector S of CS, the number of pixels NS could be smaller,
equal or greater than the number of pixels needed to obtain proportion colour harmony. If greater, pixels of S must be assigned to
other sectors (respectively if smaller, pixels of other sectors must be
assigned to S). Only the case in which the number of pixels of S is
greater than the number of pixels needed to obtain a colour harmony
is detailed here. Both cases (greater and smaller) are presented in
the pseudo-code (figure 4). To assign pixels to other sectors, we
search for the sector S � which needs pixels to verify equation 1 and
minimizes the arc-length distance. Note that the new assignments
always preserve equation 1 for S � . So S � receives the number of
pixels equal to the minimal value between what sector S � can receive and what sector S can give. In parallel a list of assignments,
called LA is updated. This operation is repeated until equation 1 is
verified for each S. So at the end of this process, LA contains the
set of pixel movements from one sector to another. The number of
pixels is equal to given or received in the pseudo-code.
The find harmony pseudo-code function given in figure 4 where
find down sector (or find up sector) returns the sector
S � which needs more (respectively less) pixels to verify equation 1
and minimizes the arc-length distance. The values of given and
received are also computed in these functions. As S � receives
the minimal value between what sector S � can receive and what
sector S can give (check), both given and received are in the
interval [0, check]. For a sector S the update(S, x) function
adds the value x to Ns .
3.4
Pixels reassignments
Once the list of assignments LA has been created, we can recolour
the image by shifting the hues of pixels as described in LA. Each
element of LA is composed by a sector, named F rom, that gives N
pixels to a sector named T o. As we choose the arc-length distance
as a cost function, one can easily understand that we reassign the N
pixels of F rom closest to T o. Note that when pixels of F rom are
at the beginning (respectively at the end), the assignment is implemented clockwise (respectively counterclockwise) on our harmonic
colour wheel. There are two possibilities to achieve this operation
(see figure 5):
• The first one consists in moving the group of N pixels directly from F rom to T o. As previously mentioned in section 3.2, the Ls lists have been sorted and this operation is
implemented in a few operations: the N pixels of F rom are
at the beginning (respectively at the end) and should be placed
at the end (respectively at the beginning) of T o; NS is updated
for F rom and T o. We call this kind of movement: JUMP.
For example if we have to assign N pixels from Lo to Lv
(considering the sector default configuration), the first N pix-
find harmony(CS)
for S in CS
check = equation 1(S)
while (check > 0)
S � = find down sector(S, given)
check = check − given
update(S, -given)
update(S � , +given)
add new assignment(LA, S, S � , given)
endwhile
while (check < 0)
S � = find up sector(S, received)
check = check + received
update(S, +received)
update(S � , -received)
add new assignment(LA, S � , S, received)
endwhile
endfor
return LA
end
Figure 4: pseudo-code to find a proportion solution.
els in Lo are moved to the end of Lv (clockwise) and No
decreases by N while Nv increases by N .
• The second algorithm consists in moving N pixels from a sector to its neighbour sector in DS that minimizes the arc-length
distance to T o. This process is repeated until N pixels have
been moved to T o. This kind of movement is called SLIP.
Observe that in case of DS = CS, SLIP produces exactly the
same reassignments as JUMP.
For example, see figure 5, if we have to move a group N of
Lo into Lv (clockwise) and DS = �Y� O� R� V� B� G), we
are going to pass through Lr (and not through the yellow,
green, and blue which is a longer way). So N pixels of Lo
are assigned to Lr and then N pixels of Lr are placed in Lv .
Note that the group of pixels of Lo assigned to Lr may be different from the pixel group of Lr assigned to Lv . In case of
DS = �Y� O� V� B� G), the group of pixels of Lo are directly
assigned to Lv .
75◦
75◦
40◦
150◦
G
Y
B
V
250◦
O
R
40◦
150◦
G
10◦
Y
B
340◦
O
R
V
10◦
340◦
250◦
Figure 5: left: JUMP from Orange to Violet sectors; right:
from Orange to Violet through Red sector
SLIP
After this step, some pixels have been reassigned, and we explain
in the next section how we assign a colour to each of these pixels.
3.5
The shifting of colour
We should assign a colour to each pixel that has been reassigned
from F rom to T o. Hereafter, P �x� y� h� s� v) refers to a pixel of
F rom and P � �x� y� h� � s� � v � ) refers to the same pixel in T o. The
difficulty lies in the choice of the ’good’ colour which is not going
to create an artifact in the picture. As Cohen-Or [Cohen-Or et al.
2006] state, colour harmony is affected by the hue channel. Observe that solutions proposed by Tokumaru et al. [Tokumaru et al.
2002] use tone distribution functions for the values of the S and V
channels which can be adapted to our model. As our model refers
to Itten, we choose a solution based only on hue. To avoid tone
disturbance, the saturation and value are preserved so s� = s and
v � = v in our model.
Four possibilities are proposed to the user:
1. Basic hue attribution (BASIC): The first naive choice consists
in attributing to P � the hue limit of sector T o that minimizes
the arc-length distance to P :
h�p =
�
T oα
T oβ
if ||hp − T oα || ≤ ||hp − T oβ ||
otherwise
Remark that if h�p is not present in T o before this operation
then a new hue is created in the image. Basic hue attribution
is realized once for all pixels in an assignment.
2. Closest hue (CLOSEST ): it assigns the existing hue in T o to
P � , which minimizes the arc-length distance to P . As the
different sectors have been sorted by hue, it is necessarily the
first or the last hue in T o. It can be written as:
h�p =
�
h0 of T o
hN−1 of T o
if ||h0 − hp || ≤ ||hN−1 − hp ||
otherwise
This time, we assure that only hues existing before reassignment are used in the produced image. Closest hue attribution
is realized one time for all pixels in an assignment.
3. Preserving ratio distance to central hue of the sector (DIS TANCE ): it measures the ratio distance between the hue of
P and the central hue of F rom and the arc-length distance of
F rom, then reports this ratio to sector T o.
h�p
||T oβ − T oα ||
= T oβ − �hp − F romα ) ×
||F romβ − F romα ||
Note that a new hue can be created using this possibility. D IS TANCE should be computed for all pixels.
4. Maintaining density distribution (DENSITY) : in that case, we
search for the density of the hue hp in F rom, then h�p receives
the hue of T o with the closest density. If more than one candidate is available, the arc-length distance is used to choose
the best one. For this choice, the list Ls is no more sorted by
hue but by density to improve computation time. Note that
the density of each sector can be computed when the image
is loaded or when sector parameters are modified by the user.
Density must be recomputed when the sector is modified by
this process. D ENSITY should be computed for all pixels.
4
Results
This section presents the results produced with our proportion harmonic model. Our proportion colour harmonization process is suitable for all images including images with atmosphere and stylized
images. It can produce images with harmony as well as with different stylizations too. This model is well adapted to the general user,
designer and artist.
The choice of the six sectors is important. As previously mentioned,
the sector definition is a crucial step. For example, if the user defines the red interval in the orange hues with Itten’s proportions, the
result could not be harmonious according to Itten’s definitions. The
joined pictures illustrate results obtained with our tool.
Figures 7 and 12 use default sectors values of DS
�Y� O� R� V� B� G). (cf. subsection 3.1).
=
To illustrate the different possibilities of our tool, figure 7
presents all the combinations to produce colour harmony with
CS = �O� R� G). The top image of figure 7 is the original
image of a bridge. This photograph is a natural image without
atmosphere. The images on the two following lines show the
two pixel reassignment methods (JUMP on the first line, SLIP on
the second). The columns show the four colour shifting methods
(BASIC on the left, then CLOSEST , DISTANCE and DENSITY on the
right). In this image composed by 196 608 pixels (512×384), with
CS = �O� R� G) we have No = 11957, Nr = 912, Ng = 33756
so the pixels number to be considered is 46625. Harmonic colour
proportion is obtained if No = 11656, Nr = 17484, Ng = 17484.
In that case, orange and green sectors give respectively 301 and
16271 pixels to the red one.
The CPU time needed to compute a harmony on an Intel 2.53GHz
with 4Go of memory, depends naturally on the size of the picture
but is influenced mostly by the colour shifting operation (see table 2). Using SLIP and DENSITY methods computation time is
1.59s. This is due to the green colour which give pixels to the red
one passing through two other sectors. Each time we have to pass
through a new sector, we have to recalculate pixel’s density of this
sector.
Assignment type
Jump
Slip
Basic
0.05
0.05
Closest
0.04
0.03
Distance
0.18
0.18
Density
0.17
1.59
Table 2: Computation time in seconds of Figure 7
Even if the results seem to be acceptable, we can observe that JUMP
creates dark red areas with BASIC, CLOSEST or DISTANCE methods. This is due to the fact that mainly red colours are created
in these resulting images. Only DENSITY preserves the main red
colour used in the original image.
As one can see, a lot of pixels have to be given to the red colour.
Given Itten’s proportions, red should be as important as green.
Green is very present in this picture (grass and trees), while we
have few red. Note that the JUMP method creates artifacts in the
grass. This is due to the green pixels which are directly assigned
to the red colour. In this example, artifacts are significant because
red and green are opposed on the chromatic hue wheel. Note that
this problem disappears using the SLIP method which prevents the
creation of additional red areas because pixel colour is changed step
by step, colour by colour. As indicated above, this is not necessarily the same pixels (depending on the shifting method) which are
shifted from one colour to another. The red and orange are more
vivid and light red areas are created on the roof of the bridge to
preserve the proportions. As DS is composed by the six sectors,
colour proportion harmony is produced using more attenuated hue
variations. Image variations can be produced with SLIP regarding
DISTANCE or DENSITY as seen on the low wall at the foreground.
Despite the fact that the SLIP method is less brutal than the JUMP
method, one can deduce that the probability to create artifacts is
high when new colours are created or if the presence of a colour is
largely insufficient with respect to the quantity it should have.
To illustrate artifacts generation, we propose hereafter a particularly case. Left part of figure 6 presents a segmented image. Each
area has the same size. We apply the JUMP and CLOSEST methods. As one can see, there is no noise in the resulting image (right
of the figure). This is due to the way our pixels are basically
listed (from bottom left to top right). Harmony is computed with
CS = �Y� R� B� G). Following Itten’s proportions, yellow should
be the smallest area in this image. So, some of this yellow pixels
should be given equally to red and green colours and more should
be given to blue colour which has the biggest proportion. Remark
that due to the way used to list pixels, blue and green areas are on
the top of the yellow area and the red one is at the bottom.
Figure 6: left: original; right: harmony with CS=(Y,R,B,G).
Figure 11 presents a stylized painting from comics creators and illustrators (figure on the left). It contains a threatening atmosphere
in ochres and brown colours. Harmony has been realized with DEN SITY and JUMP methods with CS = �Y� O� R� V ). As this picture
contains 390400 pixels with no blue nor green, the harmony computation is applied to the entire image. There are 8146 yellow pixels,
381637 orange pixels, 615 red pixels and 1 violet pixel. According to Itten’s colour proportions, these colours need respectively
+45090 pixels, -310656 pixels, +105857 pixels and +159708 pixels. The picture on the right of figure 11 demonstrates that our
model works well even when the number of reassignments is important (80% of the image). Note that the difference between the
two images is subtle: the resulting image is redder than the original one. As mentioned before, an artistic hand-made image should
be almost harmonious. This is why in this picture the difference is
subtle even if 80% of the pixels have their colour shifted. As one
can see in this example, our tool is well suited to help artists to improve the colour harmony of their creations, especially since, like
artists would do, we use the existing colours.
We have also realized benchmarks for this image. As mentioned,
80% of the pixels have been reassigned. Table 3 presents results.
As the computation time depends on the number of assignments
in LA for BASIC and CLOSEST methods, we have naturally almost
the same computation time in tables 2 and 3. The number of pixels influence mostly computation time for DISTANCE and DENSITY
methods. Note that with SLIP and DENSITY computation time is
similar to JUMP and DENSITY. In fact, for this case, each reassignment is realized between neighbour sectors so JUMP and SLIP
methods are equivalent.
Assignment type
Jump
Slip
Basic
0.07
0.08
Closest
0.09
0.09
Distance
0.23
0.23
Density
0.78
0.82
Table 3: Computation time in seconds of Figure 11
Figure 9 is a photograph of a beach. The left image is the original photograph. In the picture at the center of the figure (CS =
�R� G)), we can see that the sand and the rock have become redder, and the vegetation at the foreground gives the impression (as
improbable as it may be in reality) of a nice autumn day. The last
image has been computed with CS = �O� R� G). We can see that
this is almost the same image but the colours are softer than in the
previous image. This is due to the new colour added in CS which
permits to slip one more time between red and green colour sectors. A semantic or at least a region segmentation of the image
would help to give priorities and would prevents artifacts.
Figure 12 is an image of tree tops with a green dominant colour.
The top image is the original photograph. In the first picture, harmony is computed with CS = �B� V ). We use a SLIP movement
and CLOSEST method to choose the placement of the colours in the
sectors: every shifted pixel in the new colour has the first existing
hue. The same methods are used for the second image but harmony is computed on orange and green colours so CS = �O� G).
The third picture is a harmony on all colours of DS except orange:
CS = �Y� R� V� B� G). The reassignment of pixels is the same as
the previous one but the DISTANCE method is used for the shifting.
Note that most part of the image becomes violet. This is due to the
Itten proportion harmony where violet is the bigger proportion. In
the last picture, CS = �Y� V ) and we use DENSITY and JUMP. As
one can see, when a colour harmony is generated, new colours can
be created, particularly using SLIP movement and when DS defines
existing hues in the image. The resulting image is more stylized especially if objects in the scene cannot have these colours in real life
(see first picture on the last line of figure 12). We can deduce that
choosing colours not represented in a natural image can create stylized image and/or atmosphere depending on the given proportions.
Figure 10 is a stylized picture with a violent atmosphere with yellow and orange colours from the illustrator of the comics “Gilgamesh”. The left picture is the original image and the right picture has been computed with DENSITY and JUMP and with CS =
�Y� O) the most important colours in the image. As one can see,
torches are less luminous in this image: orange colour proportion
is bigger than yellow proportion. So some of the yellow pixels are
changed in orange. Yellow and orange pixels represent 64% of the
image colours and 35% of these colours have been shifted. One
can see again that the difference is more subtle on an artistic picture
than on a natural image.
Our algorithm is also able to harmonize a given part of the image.
The user desaturates the unwanted parts and only the saturated pixels are taken into account (see figure 8). The left image of this figure
is the original photograph. The middle image represents only the
lower of the original image. The white part at the top is a desaturated colour and is not taken into account in the harmony process.
The same image has been created with a white part at the bottom
of the original image. Thus we have decomposed the original image into two different images. The last image shows a composition
of those two images, where each one is the result of two different
harmonies. The top part has been computed with DISTANCE and
SLIP methods, DS = �Y� O� R� B� G) and CS = �Y� R� B� G).
The bottom has been computed with DISTANCE and JUMP methods
with CS = �R� V ).
Remark that if our system is able to harmonize an image or only a
part of it, it is also able to harmonize an image created from two
different ones.
5 Conclusion
We have presented our proportion harmonization model. It is userdefinable since the default colour sectors and proportions can be
modified.
It is designed to help professional or casual users who want to obtain colour proportion harmony. It can be applied to photographs,
images with atmosphere, part of images or to create stylized images or to compose scenarios. Different possibilities have been presented to assign pixels from a sector to another one (JUMP, SLIP)
and to attribute its new colour (BASIC, CLOSEST , DISTANCE and
DENSITY ). The creation or the preservation of atmosphere by a
definition of particular sectors is one of the new capabilities, never
produced before, and easy to use. By the way, currently this tool is
in experimental use by some of our local artists.
Future work will try to integrate saturation and value transformations to adjust or enhance visual colour contrasts. In addition, even
if the user can leave certain regions unchanged by masking out these
regions, it would be nice if he could specify region priorities to apply reassignment methods. Moreover, a semantic segmentation of
the image would help to give priorities.
References
A DOBE K ULER, 2006. http://kuler.adobe.com.
ATENCIA , A., B OYER , V., AND B OURDIN , J.-J. 2008. From
detail to global view, an impressionist approach. In IEEE SITIS.
C HEVREUL , M. E. 1839. De la loi du contraste simultané des
couleurs et de l’assortiment des objets colorés. Paris.
C OHEN -O R , D., S ORKINE , O., G AL , R., L EYVAND , T., AND
X U , Y.-Q. 2006. Color Harmonization. ACM Transactions on
Graphics (Proceedings of ACM SIGGRAPH) 25, 3, 624–630.
C OLOR S CHEMER, 2000. http://www.colorschemer.com.
G RANVILLE , W. C., AND JACOBSON , E. 1944. Colorimetric
specification of the color harmony manual from spectrophotometric measurements. Journal of Optical Society of America 34,
7, 382–395.
I TTEN , J. 1961. Kunst der Farbe. Ravensburg: Otto Maier Verlag.
L I -C HEN , U ., AND RONNIER L UO , M. 2006. A colour harmony
model for two-colour combinations. In Color Reasearch and
Application.
M AXWELL , C. 1860. On the theory of compound colours. In
Philosophical Transactions, no. 150, 57–84.
M UNSELL , A. H. 1921. A Grammar of Colors. Mittineague :
Strathmore Paper Company.
NACK , F., M ANNIESING , A., AND H ARDMAN , L. 2003. Colour
picking: the pecking order of form and function. In ACM Multimedia, 279–282.
O STWALD , W., AND B IRREN , F. 1969. The Color Primer. New
York: Van Nostrand Reinhold Company.
S AUVAGET, C., AND B OYER , V. 2008. Comics stylization from
photographs. In ISVC (1), 1125–1134.
S AWANT, N., AND M ITRA , N. J. 2008. Color harmonization for
videos. In ICVGIP, 576–582.
T OKUMARU , M., M URANAKA , N., AND I MANISHI , S. 2002.
Color design support system considering color harmony. In Proceedings of the IEEE International Conference on Fuzzy Systems, 378–383.
VON .
G OETHE , J. W. 1810. Zur Farbenlehre. Stuttgart: Cotta’sche
Buchhandlung.
W ESTLAND , S., L AYCOCK , K., C HEUNG , V., H ENRY, P., AND
M AHYAR , F. 2007. Colour harmony. In SDC : Society of Dyers
and Colourists.
Figure 7: top: original image; center line: JUMP movement with BASIC, CLOSEST , DISTANCE , DENSITY from left to right; last line:
movement with BASIC, CLOSEST , DISTANCE , DENSITY from left to right.
SLIP
Figure 8: left: original image; center: image area considered (white desaturated part is not taken into account); right: a different harmony
on the two defined parts of the image.
Figure 9: left: original image; center: harmonization on red and green colours; right: harmonization on orange, red and green colour.
Figure 10: left: original image (part of a picture made by the illustrator Alain Brion for the comics book Gilgamesh); right: harmonization
on yellow and orange colours.
Figure 11: left: original image (Photograph by Yann Minh of a live painting realized by J. Bodin, A. Brion, G. Francescano and E. Scala
during the Zone Franche show in 2009); right: harmonization on yellow, orange, red and violet colours.
Figure 12: top: original image; 1st: orange and green harmony with CLOSEST and SLIP; 2nd: same methods on green and blue colours;
3rd: DISTANCE and SLIP methods with all colours except orange; 4th: DENSITY and JUMP with yellow and purple colours.