http://www.colourtech.org
Stephen Westland
Centre for Colour Design Technology
University of Leeds s.westland@leeds.ac.uk
June 2005 Oxford Brookes University

Computational Colour Vision
Introduce some basic concepts - the physical basis of colour
Phenomenology of colour perception (the problem)
Computational approaches to how colour vision works
Computational and psychophysical studies of transparency perception

The Physical Basis of Colour
C( l ) = E( l )P( l )
The colour signal C( l ) is the product at each wavelength of the power in the light source and the reflectance of the object
E( l )
P( l )
E( l )P( l )

Cone spectral sensitivity
S
M
L
∫ l )P( l ) F
L
( l ) dl
∫ l )P( l ) F
M
( l ) dl
∫ l )P( l ) F
S
( l ) dl

Cone Responses
Each cone produces a univariant response
∫ l )P( l ) F
L
( l ) dl
∫ l )P( l ) F
M
( l ) dl
∫ l )P( l ) F
S
( l ) dl
S M L
Colour perception stems from the comparative responses of the three cone responses
Colour is a perception – ‘the rays are not coloured’

Colour Constancy
Objects tend to retain their approximate daylight appearance when viewed under a wide range of different light sources
P Indoors (100 cd/m 2 ) Outdoors (10,000 cd/m 2 )
0.01
1 100
0.99
99 9900
The visual system is able to discount changes in the intensity or spectral composition of the illumination
WHY? / HOW?

X

X

Computational Explanation
L
1
M
1
S
1
∫
1
( l )P( l ) F
L
( l ) dl
∫
1
( l )P( l ) F
M
( l ) dl
∫
1
( l )P( l ) F
S
( l ) dl
L
1
M
1
S
1
/ L
1W
/ M
1W
/ S
1W
= L
2
= M
2
/ L
2W
/ M
2W
= S
2
/ S
2W
L
2
M
2
S
2
∫
2
( l )P( l ) F
L
( l ) dl
∫
2
( l )P( l ) F
M
( l ) dl
∫
2
( l )P( l ) F
S
( l ) dl
D =
0
0
0
L
1W
/L
2W
M
1W
/M
2W
S
1W
/S
2W
0
0
0 e
1
= De
2 e
1
=
L
1
M
1
S
1 e
2
=
L
2
M
2
S
2

Practical Use – Colour Correction
Camera RGB values vary for a scene depending upon the light source colour correction
In order to correct the images we need an estimate of the light source under which the original image was taken brightest pixel is white grey-world hypothesis

Colour Constancy
Adaptation is too slow to explain colour constancy
“ Any visual system that achieves colour constancy is making use of the constraints in the statistics of surfaces and lights ” – Maloney (1986)
Is it possible for the visual system to recover the spectral reflectance factors of the surfaces in scenes from the cone responses?
∫ l )P( l ) F
L
( l ) dl
∫ l )P( l ) F
M
( l ) dl
∫ l )P( l ) F
S
( l ) dl

Basis Functions
P (l)  S w i
B i
( l )
Using a process such as SVD or PCA we can compute a set of basis functions B i
( l ) such that each reflectance spectrum may be represented by a linear sum of basis functions - a linear model of low dimensionality.
If we use n basis functions then each spectrum can be represented by just n scalars or weights.

0.35
0.3
0.25
0.2
0.15
0.1
0.05
400
0.55
0.5
0.45
0.4
450
0.8
0.6
0.4
0.2
0
-0.2
-0.4
400 500 550
Wavelength
600 650 700 450 500 550
Wavelength
600 650 700
Original 1 BF
P( l ) = w
1
B
1
( l )

0.3
0.25
0.2
0.15
0.1
0.05
400
0.55
0.5
0.45
0.4
0.35
450
0.8
0.6
0.4
0.2
0
-0.2
-0.4
400 500 550
Wavelength
600 650 700 450 500 550
Wavelength
600 650 700
Original 1 BF 2 BF
P( l ) = w
1
B
1
( l ) + w
2
B
2
( l )

0.3
0.25
0.2
0.15
0.1
0.05
400
0.55
0.5
0.45
0.4
0.35
450
0.8
0.6
0.4
0.2
0
-0.2
-0.4
400 500 550
Wavelength
600 650 700 450 500 550
Wavelength
600 650 700
Original 1 BF 2 BF 3 BF
P( l ) = w
1
B
1
( l ) + w
2
B
2
( l ) + w
3
B
3
( l )

How many Basis Functions are Required?
PC Variance Total
Variance
1
2
3
76.77
15.83
5.96
76.77
92.60
98.56
4
5
0.76
0.37
99.32
99.68
6
7
0.12
0.09
99.80
99.89
8 0.04 99.93
About 99% of the variance can be accounted for by a 3-D model (Maloney &
Wandell, 1986)
But what proportion of the variance do we need to account for?
6-9 basis functions are required

Simultaneous Contrast

original original covered by filter original with small filter

Colour Constancy - spatial comparisons
“For the qualities of lights and colours are perceived by the eye only by comparing them with one another” (Alhazen, 1025)
“… object colour depends upon the ratios of light reflected from the various parts of the visual field rather than on the absolute amount of light reflected” (Marr) e i,1
/e i,2 i = {L, M, S}
= e' i,1
/e' i,2
(Foster)
Ratio under second light source
Ratio under first light source

Spatial Comparison of Cone Excitations
Retinex – Land and McCann (1971)
Foster and Nascimento (1994)
L
1
L
2
=k
L’
1
L’
2
=k

Transparency Perception
(Ripamonti and Westland, 2001) e’
1 e
1 e’ e
2
2 e
1
/e
2
= e’
1
/e’
2

What is transparency?
An object is (physically) transparent if some proportion of the incident radiation that falls upon the object is able to pass through the object.

What is perceptual transparency?
Perceptual transparency is the process ‘of seeing one object through another’ (Helmholz, 1867)
Physical transparency is neither a necessary or sufficient condition for perceptual transparency (Metelli, 1974)
Even in the complete absence of any physical transparency it is possible to experience perceptual transparency

Perceptual transparency

Research Questions
What mechanisms could drive perceptual transparency?
What are the chromatic conditions that cause transparency?
Could transparency and colour constancy be linked?

Perceptual transparency

Transparency and Spatial Ratios e i,1 e i,2
T( l ) e' i,1 e' i,2 e i,1
/e i,2
= e' i,1
/e' i,2

Experimental
Computational analysis to investigate whether for physical transparency the cone ratios are preserved
Psychophysical study to investigate whether the invariance of spatial ratios can predict chromatic conditions for perceptual transparency
Psychophysical study to compare the performance of the ratio-invariance model when the number of surfaces is varied

Physical Model of Transparency
(1-b)PT 2 (1-b)bP 2 T 4
(1-b)b 2 P 3 T 6 b
T opaque surface P
P'( l ) = P( l )[T( l )(1-b) 2 ] 2 (Wyszecki & Stiles, 1982)

Monte Carlo Simulation
1. A pair of surfaces P
1
( l ) and P
2
( l ) were randomly selected
2. A filter was randomly selected (defined by a gaussian distribution)
3. The cone excitations were computed for the surfaces viewed directly (under D65) and through the filter
4. Steps 1-3 repeated 1000 times
P
1
( l ) P
2
( l ) s e i,1
/e i,2 e' i,1
/e' i,2 l m

Monte Carlo Results e i ,1 e i
'
,1 e i
'
,2 e i ,2 e i ,1
/ e i ,2

Monte Carlo Results e i ,1 e i
'
,1 e i
'
,2 e i ,2 filter S cones M cones L cones s
= 10nm 0.9988 (0.9968) 0.9964 (0.9951) 0.9996 (0.9955) s
= 50nm 0.9978 (0.9231) 1.0037 (0.8666) 0.9788 (0.8599) s
= 200nm 0.9231 (0.8032) 0.8885 (0.7154) 0.9144 (0.7284)
The ratios are approximately invariant
Invariance is slightly better for the S cones
Invariance decreases as the spectral transmittance decreases

Convergence x
A x
P x
Q x
B x
A x
P x
B x
Q g x
P x
Q
= a x
A
= a x
B
+ (1a ) g
+(1a ) g
Da Pos, 1989, D’Zmura et al ., 1997

Psychophysical Stimuli I
(a) convergent
(deviation  0)
(b) invariant
(deviation = 0) deviation i
= 1 - [e i,1
/e i,2
]/[ e' i,1
/e' i,2
]

Psychophysical Results I
5
3 d'
1
-1
-3
0
L
0.1
M
0.2
0.3
LMS deviations
0.4
S Log. (L) Log. (S)
0.5
0.6
Log. (M) d'<0 indicates subjects' preference for convergent filter; d'=0 no preference; d'>0 indicates subjects' preference for invariant filter

Psychophysical Stimuli II

Psychophysical Results II
5
4
3 d' 2
1
0
-1
0 y = 1.1 Ln (x + 2.23)
2 4 6 8 10 12 14 number of surfaces
L
M
LMS

Conclusions
Computational and pyschophysical studies show that the invariance of cone-excitation ratios may be a useful cue driving transparency perception
Colour constancy and transparency perception may be related. Could they result from similar mechanisms, perhaps even similar groups of neurones?
There are still a mass of unsolved problems in computational colour vision including the relationship between cone excitations and the actual sensation of colour

x
A x
B x
P x
B x
Q x
A x
Q x
P g x
P x
Q
= a x
A
= a x
B
+ (1a ) g
+(1a ) g x
P x
Q
= a x
A
= a x
B
Cone excitations are transformed by a a diagonal matrix whose diagonal elements are all equal

x
A x
P x
Q x
B x
P x
Q
= b x
A
= b x
B
Cone excitations are transformed by a a diagonal matrix whose diagonal elements are not necessarily all equal
The two models can be made to be the same if the convergence model has no additive component and if the invariance model has equal cone scaling