Information Theory and Aesthetics. Application to the
Evolution of Van Gogh’s Artwork
Pembroke College, Oxford
Jaume Rigau, Miquel Feixas and Mateu Sbert
Graphics and Imaging Laboratory, University of Girona, Spain
gilab.udg.edu
Imaging
GILab’s logo
Laboratory
Graphics &
March 2015
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Preliminary
Painter
Imaging
GILab’s logo
I
Paintings by Vincent van Gogh (1853-1890), public domain
I
Full set of 800 paintings
I
Digital photographic reproductions by David Brooks, The Vincent van
c 1996-2008 David Brooks
Gogh Gallery Laboratory
Graphics &
Self-Portrait with Bandaged
Ear and Pipe, 1889
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Preliminary
Imaging
GILab’s logo
Preliminary
Informational Aesthetic Measures
Previous Work
Basic Tools
Three Aesthetic Measures
Informational Analysis
Van Gogh Periods
Color Mutual Information
Definition
Van Gogh Periods
Entropy-Rate
Definition
Van Gogh Periods
Conclusions
Acknowledgments
More Info...
Pink Peach Tree in Blossom, 1888
Laboratory
Graphics &
Outline
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Aesthetic Measures
Previous Work
Birkhoff’s Aesthetic Measure
The origins
George David Birkhoff, 1884-1944
Photograph by Bachrach
c 2008 National Academy of Sciences
Definition
I
[Birkhoff 33]
Ratio order and complexity
I
O
C
Impossibility of comparing objects of different classes
I
The aesthetic experience depends on each observer
Imaging
GILab’s logo
Laboratory
Graphics &
M=
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Aesthetic Measures
Previous Work
Bense’s Model
The birth of Informational Aesthetics
Max Bense, 1910-1990
c 2006 Universität Stuttgart
Artistic process
[Bense 69]
I
A repertoire of elements is transmitted to the final product
I
The creative process is a selective process
I
Order is produced from disorder
1. Initial repertoire: range of colors
2. Palette: selected repertoire
Imaging
GILab’s logo
Laboratory
Graphics &
3. Spatial distribution: palette → physical support
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Aesthetic Measures
Previous Work
Bense’s Model
The birth of Informational Aesthetics
Max Bense, 1910-1990
c 2006 Universität Stuttgart
Artist
representation
to code
Imaging
GILab’s logo
Rep1
Laboratory
Graphics &
Description
Product
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Aesthetic Measures
Previous Work
Bense’s Model
The birth of Informational Aesthetics
Max Bense, 1910-1990
c 2006 Universität Stuttgart
Artist
representation
to code
Product
recognition
to decode
Observer
Rep1 ∩ Rep2
Description
Rep1
Rep2
Evaluation
GILab’s logo
Laboratory
Graphics &
Art is communication!
Imaging
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Aesthetic Measures
Basic Tools
Complexity and Order
How to measure them?
Conceptualizing Birkhoff’s aesthetic measure
following Bense’s perspective
I
Heterogeneity of the palette: Shannon entropy
I
Information transfer: Mutual information
I
Descriptive complexity: Kolmogorov complexity
Imaging
GILab’s logo
Laboratory
Graphics &
Informational aesthetics framework
Complexity and order interpretations
Normalized measures of M
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Aesthetic Measures
Basic Tools
Shannon Entropy
The basis of Information Theory
Claude Elwood Shannon, 1916-2001
c 2006-2008 Alcatel-Lucent
Information
I
Discrete stochastic variable: X = {x1 , . . . , xn }
I
Probability distribution: p(X ) = {p(x1 ), . . . , p(xn )}
I
Surprise or information: S(xi ) = − log p(xi )
Discrete Shannon entropy
[Shannon 48]
H(X ) = EX (S(X )) = −
X
pi log pi bits
GILab’s logo
Laboratory
Graphics &
x∈X
Imaging
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Aesthetic Measures
Basic Tools
Information Channel
A discrete memoryless channel
Loss
Imaging
GILab’s logo
Sender
In
Out
Receiver
Laboratory
Graphics &
X
Noise
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Y
Informational Aesthetic Measures
Basic Tools
Information Channel
A discrete memoryless channel
Loss
X
Sender
Noise
In
Out
p(X )
Receiver
p(Y |X )
p(Y )
X →Y
Information transfer between X and Y
Mutual Information
[Shannon 48]
I (X , Y ) =
XX
p(x, y ) log
Imaging
GILab’s logo
Laboratory
Graphics &
x∈X y ∈Y
p(x, y )
bits
p(x)p(y )
Dependence or correlation between X and Y
Information Theory and Aesthetics
Y
Info-Metrics Institute 2015 Spring Workshop
Informational Aesthetic Measures
Basic Tools
Information Channel
A discrete memoryless channel
Loss
X
Sender
Noise
In
p(X )
Out
Receiver
p(Y |X )
X →Y
Entropy of X conditional on Y
Discrete conditional Shannon entropy
H(X |Y ) = −
XX
[Shannon 48]
p(x, y ) log p(x|y ) bits
Imaging
GILab’s logo
Laboratory
Graphics &
y ∈Y x∈X
Remaining uncertainty of X known Y
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Y
p(Y )
Informational Aesthetic Measures
Basic Tools
Mutual Information
Venn diagram
Entropy relation
I (X , Y ) = H(X ) − H(X |Y )
= H(Y ) − H(Y |X )
H(X ) = I (X , Y ) + H(X |Y )
H(X )
H(Y )
Imaging
GILab’s logo
Laboratory
Graphics &
H(X |Y ) I (X , Y ) H(Y |X )
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Aesthetic Measures
Basic Tools
Kolmogorov Complexity
A descriptive complexity
Andrey Nikolayevich Kolmogorov, 1903-1987
c 1966 Novosti Press Agency
K
[Kolmogorov, Chaitin, and Solomonoff 60’s]
Length of the shortest binary program to compute x:
K (x) = minU(p)=x {`(p)}
Imaging
GILab’s logo
Laboratory
Graphics &
Algorithmic information / randomness
Length of the ultimate compressed version of x
Non-computable ⇒ approximated by standard real-world compressors
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Aesthetic Measures
Three Aesthetic Measures
Image Data
Color histograms
Image I of N pixels
Histogram H = f (I, bins)
Palette Represented by
Tristimulus sRGB color system
Alphabet Xrgb = {0, . . . , 255}3
Luminance ITU-R Recommendation BT.709
Alphabet X` = [0, 255]
HSV sRGB representation = (hue, saturation, value)
Alphabet XHSV = {0, . . . , 360} × {0, . . . , 1}2
Imaging
GILab’s logo
Laboratory
Graphics &
Color (C, C ) = [(Xrgb , Xrgb )|(X` , X` )|(XHSV , XHSV )]
Region (R, R)
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Aesthetic Measures
Three Aesthetic Measures
Bense’s Perspective
Based on Shannon entropy
Redundancy
Maximum entropy − Entropy of the palette
Hmax − H(C )
C = Xrgb ⇒ n = |Xrgb | = 2563 ⇒ Hmax = log n = 24
C = X` ⇒ n = |X` | = 256 ⇒ Hmax = log n = 8
I
Order: redundancy
I
Complexity: maximum entropy
I
Interpretation: reduction of uncertainty due to
the choice of a given palette
Imaging
GILab’s logo
MB =
Hmax − H(C )
Hmax
Laboratory
Graphics &
Aesthetic measure
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Aesthetic Measures
Three Aesthetic Measures
Kolmogorov’s Perspective
Based on algorithmic information
Compression ratio
I
Quantify the reduction in data size produced by a data compression
algorithm
I
More structure or regular patterns ⇒ order↑ ⇒ compression↑
N = number of pixels
Imaging
GILab’s logo
I
Order: algorithmic reduction of the data size
I
Complexity: data size using a constant code
I
Interpretation: degree of order without any a
priori knowledge on the palette
MK =
NHmax − K (I)
NHmax
Laboratory
Graphics &
Aesthetic measure
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Aesthetic Measures
Three Aesthetic Measures
Channel Perspective
Self-Portrait in Front of the Easel, 1888
Wheat Field with Crows, 1890
The creative channel
Imaging
GILab’s logo
Laboratory
Graphics &
The creative process described by Max Bense (1969) can be understood as
the realization of an information channel between the palette and a set of
regions of the image.
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Aesthetic Measures
Three Aesthetic Measures
Channel Perspective
p(C )
p(R)
C
R
p(R|C )
Histogram
Image Regions
Imaging
GILab’s logo
I
C is the input r.v. associated with the set of bins C of H.
I
R is the output r.v. associated with the set of regions R of I.
I
p(C ) is the probability of bin c ∈ C.
I
p(R) is the normalized area of region r ∈ R.
I
p(R|C ) is the transition probability between c and r .
Laboratory
Graphics &
Image information channel
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Aesthetic Measures
Three Aesthetic Measures
Channel Perspective
p(C )
p(R)
C
R
p(R|C )
Histogram
Image Regions
Mutual information
Shared information or correlation between C and R:
I (C , R) =
XX
Imaging
GILab’s logo
p(c, r )
p(c)p(r )
Laboratory
Graphics &
c∈C r ∈R
p(c, r ) log
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Aesthetic Measures
Three Aesthetic Measures
Channel Perspective
Looking for the compositional structure
Image decomposition
[Rigau et al. 05]
I
A partitioning algorithm builds a region-tree guided by I (C , R)
I
Discrete information channel C → R
I
Acquisition of information increases the mutual information and
decreases uncertainty:
H(C ) = I (C , R)↑ + H(C |R)↓
The tree captures the structure and hierarchy of the image
Imaging
GILab’s logo
Laboratory
Graphics &
The I (C , R)↑ quantifies the capacity of an image to be ordered
or the feasibility of decomposing it by an observer
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Aesthetic Measures
Three Aesthetic Measures
Channel Perspective
Looking for the compositional structure
Aesthetic measure
I
Order: degree of I captured by a decomposition
I
Complexity: initial uncertainty (entropy)
I
Ratio of I :
I
Image complexity:
Ms (n) =
Ms-1
I (C , R)
H(C )
I (C , R)
H(C )
=n
Imaging
GILab’s logo
Laboratory
Graphics &
The number of regions n for a ratio of I
is a measure of image complexity
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Aesthetic Measures
Three Aesthetic Measures
A Simple Composition
Imaging
GILab’s logo
Laboratory
Graphics &
Potato Planting, 1884
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Aesthetic Measures
Three Aesthetic Measures
A Simple Composition
Potato Planting, 1884
Imaging
GILab’s logo
Laboratory
Graphics &
-1
= 29 regions
Ms,0.25
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Aesthetic Measures
Three Aesthetic Measures
A Complex Composition
Imaging
GILab’s logo
Laboratory
Graphics &
Irises, 1889
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Aesthetic Measures
Three Aesthetic Measures
A Complex Composition
Imaging
GILab’s logo
Laboratory
Graphics &
Irises, 1889
-1
Ms,0.25
= 3, 378 regions
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Analysis
Van Gogh Periods
Average per Period
Imaging
GILab’s logo
#
26
172
209
181
137
75
800
MB
0.422
0.486
0.384
0.351
0.342
0.334
0.388
MK
0.769
0.794
0.712
0.688
0.665
0.659
0.713
Ms-1 (0.25)
1,019
1,145
1,689
1,748
2,331
2,081
1,710
Laboratory
Graphics &
Period
Early
Nuenen
Paris
Arles
St-Rémy
Auvers
Global
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Analysis
Van Gogh Periods
Average per Period
Period
Early
Nuenen
Paris
Arles
St-Rémy
Auvers
Global
Imaging
GILab’s logo
MB
0.422
0.486
0.384
0.351
0.342
0.334
0.388
MK
0.769
0.794
0.712
0.688
0.665
0.659
0.713
Ms-1 (0.25)
1,019
1,145
1,689
1,748
2,331
2,081
1,710
Redundancy decrease → Increase color diversity (from Nuenen)
Laboratory
Graphics &
I
#
26
172
209
181
137
75
800
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Analysis
Van Gogh Periods
Average per Period
Imaging
GILab’s logo
#
26
172
209
181
137
75
800
MB
0.422
0.486
0.384
0.351
0.342
0.334
0.388
MK
0.769
0.794
0.712
0.688
0.665
0.659
0.713
Ms-1 (0.25)
1,019
1,145
1,689
1,748
2,331
2,081
1,710
I
Redundancy decrease → Increase color diversity (from Nuenen)
I
Order decrease → complexity (from Nuenen)
Laboratory
Graphics &
Period
Early
Nuenen
Paris
Arles
St-Rémy
Auvers
Global
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Analysis
Van Gogh Periods
Average per Period
Imaging
GILab’s logo
#
26
172
209
181
137
75
800
MB
0.422
0.486
0.384
0.351
0.342
0.334
0.388
MK
0.769
0.794
0.712
0.688
0.665
0.659
0.713
Ms-1 (0.25)
1,019
1,145
1,689
1,748
2,331
2,081
1,710
I
Redundancy decrease → Increase color diversity (from Nuenen)
I
Order decrease → complexity (from Nuenen)
I
Compositional complexity increase (before Auvers)
Laboratory
Graphics &
Period
Early
Nuenen
Paris
Arles
St-Rémy
Auvers
Global
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Analysis
Van Gogh Periods
Average per Period
Imaging
GILab’s logo
#
26
172
209
181
137
75
800
MB
0.422
0.486
0.384
0.351
0.342
0.334
0.388
MK
0.769
0.794
0.712
0.688
0.665
0.659
0.713
Ms-1 (0.25)
1,019
1,145
1,689
1,748
2,331
2,081
1,710
I
Redundancy decrease → Increase color diversity (from Nuenen)
I
Order decrease → complexity (from Nuenen)
I
Compositional complexity increase (before Auvers)
Laboratory
Graphics &
Period
Early
Nuenen
Paris
Arles
St-Rémy
Auvers
Global
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Analysis
Van Gogh Periods
Earliest Paintings
Fisherman’s Wife on the Beach, 1882
GILab’s logo
Laboratory
Graphics &
-1
(MB ,MK ,Ms,0.25
)=(0.418, 0.759, 1264) (0.422, 0.769, 1019)
Imaging
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Analysis
Van Gogh Periods
Earliest Paintings
Fisherman’s Wife on the Beach, 1882
MB ↑ Limited palette
MK ↑ High degree of order
Ms-1 ↓ Basic compositions
GILab’s logo
Laboratory
Graphics &
-1
(MB ,MK ,Ms,0.25
)=(0.418, 0.759, 1264) (0.422, 0.769, 1019)
Imaging
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Analysis
Van Gogh Periods
Nuenen/Antwerp
Shepherd with a Flock of Sheep, 1884
GILab’s logo
Laboratory
Graphics &
-1
(MB ,MK ,Ms,0.25
)=(0.463, 0.739, 875) (0.486, 0.794, 1145)
Imaging
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Analysis
Van Gogh Periods
Nuenen/Antwerp
Shepherd with a Flock of Sheep, 1884
MB ↑ Dark palette and dull in tones
MK ' Few colors and easy structure
Ms-1 ↓ Simple compositions
GILab’s logo
Laboratory
Graphics &
-1
(MB ,MK ,Ms,0.25
)=(0.463, 0.739, 875) (0.486, 0.794, 1145)
Imaging
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Analysis
Van Gogh Periods
Paris
The Seine with the Pont de la Grande Jette, 1887
GILab’s logo
Laboratory
Graphics &
-1
(MB ,MK ,Ms,0.25
)=(0.385, 0.718, 1396) (0.384, 0.712, 1689)
Imaging
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Analysis
Van Gogh Periods
Paris
The Seine with the Pont de la Grande Jette, 1887
MB ↓ From dark-hued to bright colors
MK ↓ More details, elements, and colors
Ms-1 ↑ Complex compositions
GILab’s logo
Laboratory
Graphics &
-1
(MB ,MK ,Ms,0.25
)=(0.385, 0.718, 1396) (0.384, 0.712, 1689)
Imaging
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Analysis
Van Gogh Periods
Arles
Sunset: Wheat Fields Near Arles, 1888
GILab’s logo
Laboratory
Graphics &
-1
(MB ,MK ,Ms,0.25
)=(0.345, 0.697, 1648) (0.351, 0.688, 1748)
Imaging
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Analysis
Van Gogh Periods
Arles
Sunset: Wheat Fields Near Arles, 1888
MB ↓ Vibrant and saturated colors
MK ↓ Exploration of complementary color contrasts
Ms-1 ↑ Complex compositions
GILab’s logo
Laboratory
Graphics &
-1
(MB ,MK ,Ms,0.25
)=(0.345, 0.697, 1648) (0.351, 0.688, 1748)
Imaging
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Analysis
Van Gogh Periods
Saint-Rémy
Olive Grove: Pale Blue Sky, 1889
GILab’s logo
Laboratory
Graphics &
-1
(MB ,MK ,Ms,0.25
)=(0.339, 0.593, 2456) (0.342, 0.665, 2331)
Imaging
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Analysis
Van Gogh Periods
Saint-Rémy
Olive Grove: Pale Blue Sky, 1889
MB ↓ Heterogeneous and bright palette
MK ↑ Landscapes and swirls
Ms-1 ↑ High compositional complexity
GILab’s logo
Laboratory
Graphics &
-1
(MB ,MK ,Ms,0.25
)=(0.339, 0.593, 2456) (0.342, 0.665, 2331)
Imaging
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Analysis
Van Gogh Periods
Auvers-sur-Oise
Daubigny’s Garden, 1890
GILab’s logo
Laboratory
Graphics &
-1
(MB ,MK ,Ms,0.25
)=(0.315, 0.714, 2375) (0.334, 0.659, 2081)
Imaging
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Informational Analysis
Van Gogh Periods
Auvers-sur-Oise
Daubigny’s Garden, 1890
MB ↓ Palette highly contrasted
MK ↓ Luminance used in a complex way
Ms-1 ' High compositional complexity
GILab’s logo
Laboratory
Graphics &
-1
(MB ,MK ,Ms,0.25
)=(0.315, 0.714, 2375) (0.334, 0.659, 2081)
Imaging
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Periods
25%
Measure Plots
Van Gogh Periods
Página 1 de 1
Informational Analysis
MB (reversed)
25%
MK (reversed)
25%
Página 1 de 1
Página 1 de 1
-1
Ms,0.15
25%
Página 1 de 1
Página 1 de 1
25%
Página 1 de 1
-1
Ms,0.20
Imaging
GILab’s logo
Laboratory
Graphics &
25%
-1
Ms,0.25
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Periods
25%
Measure Plots
Van Gogh Periods
Página 1 de 1
Informational Analysis
MB (reversed)
25%
MK (reversed)
Página 1 de 1
25%
Página 1 de 1
Origins 1 2
25%
Página 1 de 1
Maturity 4 5 6
Página 1 de 1
-1
Ms,0.15
3
Transition
25%
Página 1 de 1
-1
Ms,0.20
Imaging
GILab’s logo
Laboratory
Graphics &
25%
-1
Ms,0.25
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Color Mutual Information
Definition
Color Mutual Information
Color information distribution in the creative channel
p(C )
p(R)
C
R
p(R|C )
Histogram
Image Regions
Mutual Information
I (C , R) =
X
p(c)
c∈C
X
r ∈R
p(r |c) log
p(r |c)
p(r )
Imaging
GILab’s logo
Laboratory
Graphics &
How is the color information distributed?
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Color Mutual Information
Definition
Color Mutual Information
Color information distribution in the creative channel
p(C )
p(R)
C
R
p(R|C )
Histogram
Image Regions
Mutual Information
I (C , R) =
X
p(c)
c∈C
X
r ∈R
p(r |c) log
p(r |c)
p(r )
Imaging
GILab’s logo
Laboratory
Graphics &
How is the color information distributed?
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Color Mutual Information
Definition
Color Mutual Information
Color information distribution in the creative channel
p(C )
p(R)
C
R
p(R|C )
Histogram
Image Regions
Color Mutual Information
I (C , R) =
X
p(c)I (c, R)
c∈C
I (c, R) =
X
p(r |c) log
r ∈R
p(r |c)
p(r )
Imaging
GILab’s logo
Laboratory
Graphics &
Information or saliency associated with each color
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Color Mutual Information
Definition
Color Mutual Information
Color information distribution in the creative channel
p(C )
p(R)
C
R
p(R|C )
Histogram
Image Regions
Kullback-Leibler distance
I (c, R) = KL(p(R|c) k p(R))
KL(p k q) =
X
p(x) log
x∈X
Imaging
GILab’s logo
Laboratory
Graphics &
I (c, R)↑ High correlation between a color and a region
I (c, R)↓ Color distributed uniformly in the image
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
p(x)
q(x)
Color Mutual Information
Van Gogh Periods
Earliest Paintings
Two Women in the Woods, 1882
Imaging
GILab’s logo
Laboratory
Graphics &
-1
(MB ,MK ,Ms,0.25
,I0.25 )=(0.310, 0.650, 2020, 1.910)
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Color Mutual Information
Van Gogh Periods
Nuenen/Antwerp
Imaging
GILab’s logo
1417, 1.291)
Laboratory
Graphics &
The Potato Eaters, 1885
-1
(MB ,MK ,Ms,0.25 ,I0.25 )=(0.575, 0.850,
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Color Mutual Information
Van Gogh Periods
Paris
Imaging
GILab’s logo
1.854)
Laboratory
Graphics &
Self-Portrait with Straw Hat, 1887
-1
(MB ,MK ,Ms,0.25 ,I0.25 )=(0.295, 0.726, 1272,
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Color Mutual Information
Van Gogh Periods
Arles
Imaging
GILab’s logo
1.905)
Laboratory
Graphics &
Vase with Fifteen Sunflowers, 1888
-1
(MB ,MK ,Ms,0.25 ,I0.25 )=(0.349, 0.581, 2736,
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Color Mutual Information
Van Gogh Periods
Saint-Rémy
Imaging
GILab’s logo
1758, 1.937)
Laboratory
Graphics &
Starry Night, 1889
-1
(MB ,MK ,Ms,0.25 ,I0.25 )=(0.322, 0.594,
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Color Mutual Information
Van Gogh Periods
Auvers-sur-Oise
Imaging
GILab’s logo
1.966)
Laboratory
Graphics &
Thatched Cottages at Cordeville, 1890
-1
(MB ,MK ,Ms,0.25
,I0.25 )=(0.312, 0.592, 2095,
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Entropy-Rate
Definition
Block Entropy
Information in streams of data
L-Block
I
Chain of random variables: . . . X-2 X-1 X0 X1 X2 . . .
I
Block of L consecutive random variables: X L = X1 . . . XL
I
L-Block probability: p(x L )
L-Block Entropy
H(X L ) = −
X
p(x L ) log p(x L )
Imaging
GILab’s logo
Laboratory
Graphics &
x L ∈X L
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Entropy-Rate
Definition
Block Entropy
Information in streams of data
L-Block
I
Chain of random variables: . . . X-2 X-1 X0 X1 X2 . . .
I
Block of L consecutive random variables: X L = X1 . . . XL
I
L-Block probability: p(x L )
L-Block Entropy
H(X L ) = −
X
p(x L ) log p(x L )
x L ∈X L
L-Block Symbol Entropy
Imaging
GILab’s logo
Laboratory
Graphics &
hx (L) = H(XL |X L−1 ) ≡ H(X L ) − H(X L−1 )
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Entropy-Rate
Definition
Randomness
Based on the Shannon entropy
Entropy-Rate
I
Average amount of information (irreducible randomness) per symbol
I
Uncertainty associated with a given symbol (knowing the preceding
symbols)
I
Optimal algorithmic compression
Imaging
GILab’s logo
H(X L )
L
≡ limL→∞ hx (L)
Laboratory
Graphics &
hx = limL→∞
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Entropy-Rate
Definition
Randomness
Based on the Shannon entropy
Entropy-Rate
I
Average amount of information (irreducible randomness) per symbol
I
Uncertainty associated with a given symbol (knowing the preceding
symbols)
I
Optimal algorithmic compression
hx = limL→∞
H(X L )
L
≡ limL→∞ hx (L)
Imaging
GILab’s logo
I
Average uncertainty surrounding a pixel
I
Degree of randomness (disorder ' compression difficulty)
I
Empirical positive correlation with contrast
Laboratory
Graphics &
Aesthetic measure
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Entropy-Rate
Van Gogh Periods
Entropy-Rate per Period
Imaging
GILab’s logo
x
hH
x
5.251
4.899
5.564
5.855
5.859
5.925
5.561
hSx
s(x)
0.724
0.810
0.709
0.598
0.652
0.531
0.804
x
6.551
6.219
6.802
7.034
6.972
7.116
6.780
I
Entropy-rate ≡ contrast
I
Early → Nuenen: hx ↓
I
Nuenen → Auvers: hx ↑
I
Optimal compression
hVx
s(x)
0.443
0.602
0.469
0.305
0.322
0.309
0.502
x
6.802
6.215
6.972
7.272
7.427
7.471
6.996
s(x)
0.452
0.776
0.405
0.286
0.248
0.305
0.547
Laboratory
Graphics &
Period
Name
Early
Nuenen
Paris
Arles
St-Rémy
Auvers
Global
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Entropy-Rate
Van Gogh Periods
Entropy-Rate Example
Imaging
GILab’s logo
Sheaves of Wheat, 1890
I
x
x
C hHSV
= (5.718, 7.696, 9.279) and hHSV
= (4.156, 7.215, 5.000) B
I
x
hHSV
↑≡ high contrast (i.e., spots in foliage)
I
x
hHSV
↓≡ low contrast (i.e., sheaves and field)
Laboratory
Graphics &
The Grove, 1890
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Conclusions
Summary
I
Informational aesthetic measures
I
I
I
I
Characterization of a painting style
Study the evolution of a painter
Visualize the most informative or salient colors and elements.
Future work
I
I
Test with other artists
Study of perceptual measures
Main contribution
Imaging
GILab’s logo
Laboratory
Graphics &
Proposal of a set of tools to help to study artist’s work and discover
relevant characteristics of a painting (or painter’s style) which could go
unnoticed by the observer.
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
Acknowledgments
Imaging
GILab’s logo
I
you for your attention
I
Info-Metrics Institute and PIIP organizers for their invitation
I
David Brooks, The Vincent van Gogh Gallery, www.vggallery.com
I
Grant TIN2013-47276-C6-1-R from Spanish Government
I
2014 SGR1232 from Catalan Government
Laboratory
Graphics &
Thanks to
The Cafe Terrace on the Place
du Forum at Night, 1888
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop
More Info...
Imaging
GILab’s logo
Laboratory
Graphics &
Additional info to be found in...
Information Theory and Aesthetics
Info-Metrics Institute 2015 Spring Workshop