On the perception of Bandlimited
Phase Distortion in Natural scenes
Kedar Vilankar, Logesh Vasu and Damon Chandler
School of Electrical and Computer Engineering
Oklahoma State University
1
Importance of Phase


Oppenheim and Lim (1981) demonstrated the importance
of phase in signals.
Magnitude
Spectrum
Phase
Spectrum
Phase spectrum contributes more to the image’s visual
appearance.
2 of 51
Cells Compute Local Information

Primates V1 is dominated by complex cells.

V1 complex cells are insensitive to phase

V1 complex cells encode the magnitude information
only.

V1 complex cells are localized to receptive fields.
3 of 51
Local Magnitude is all we need

Phase information is implicitly encoded in local
magnitude.

Morrone and Burr demonstrated computation of location
of lines and edges (Phase congruency) from local
magnitude (Complex cell response).

Also, other researchers have shown only local magnitude
is required for scene recognition and categorization.
4 of 51
Local Magnitude is all we need


Morgan et. al demonstrated local phase is of lesser
importance than local magnitude.
Magnitude
uses Spectrum
local magnitude
Phase
information
to
Spectrum
HVS
global (image-wide) phase information.
5 of 51
determine
Does HVS use Local Phase ?

Signal processing perspective, local phase is important.

If HVS uses only local magnitude, then we should not see
any impact of distortion in local phase.
Local Phase
Original
Distorted
Image Image
6 of 51
Does HVS use Local Phase ?

Complex cells compute local magnitude by combining the
responses of two simple cells.

May also exist a visual mechanism to compute local
phase using simple cells.
7 of 51
Does HVS use Local Phase ?


If HVS has mechanism to compute local phase,
then do we infer global phase from
1. only local magnitude or
2. both local magnitude and local phase
8 of 51
Summary

Global phase is most important for image appearance.
(Oppenheim & Lim)

Local magnitude can implicitly encode global phase.
(Morrone et al. and Shams et al.)

Local phase is of lesser importance for image appearance

However, local phase distortion has substantial
impact on image quality.
9 of 51
Outline

Experiment
(Local Phase Contribution)

Results
(Interesting and surprising)

Discussion
(Open questions. We need help)

Algorithm
(Local magnitude and phase Distortion
Rater)

Conclusion
(Our belief)
10 of 51
Experiment

Measure the relative contribution of local magnitude and
local phase towards image appearance.

Stimuli were created by forming hybrid in complex
wavelet subbands.

Each subbands local magnitude and local phase was taken
from 2- 4 original images.
11 of 51
Experiment
Local
Magnitude
Local
Phase
Low
Frequency
12 of 51
High
Frequency
Original Images
Frequency (cyc/deg)
Experiment
Local
Magnitude
Local
Phase
Low
Frequency
13 of 51
High
Frequency
Original Images
Frequency (cyc/deg)
Experiment

Based on permutations to make hybrid images 14
combination were created.

For each combination 12 stimuli were created.

Five subjects were asked to rate how much each original
image contribute to the appearance of the stimulus.

Viewing distance : 60cm.

Five choices : 5%, 10%, 25%, 50% and 75%
14 of 51
Result : Combination 1
Local
Magnitude
Local
Phase
Low
Frequency
15 of 51
High
Frequency
Original Images
Frequency (cyc/deg)
Result : Combination 1
Local
Magnitude
Local
Phase
Low
Frequency
16 of 51
High
Frequency
Original Images
Stimulus
Frequency (cyc/deg)
Result : Combination 1
59%
Local
Magnitude
6%
Local
Phase
8%
Low
Frequency
17 of 51
27%
High
Frequency
Frequency (cyc/deg) Stimulus
Original Images
Result : Combination 1
59%
Local
Magnitude
6%
Local
Phase
27%
8%
Low
Frequency
High
Frequency
Frequency (cyc/deg) Stimulus
When across frequencies, local magnitude and
phase have non cooperative information, then
HVS relies mostly on high frequency local
magnitude information.
Original Images
18
Result : Combination 3
Local
Magnitude
Local
Phase
Low
Frequency
19 of 51
High
Frequency
Original Images
Frequency (cyc/deg)
Result : Combination 3
Local
Magnitude
Local
Phase
Low
Frequency
20 of 51
High
Frequency
Original Images
Stimulus
Frequency (cyc/deg)
Result : Combination 3
39%
Local
Magnitude
6%
55%
Local
Phase
Low
Frequency
21 of 51
High
Frequency
Frequency (cyc/deg) Stimulus
Original Images
Result : Combination 3
39%
Local
Magnitude
6%
55%
Local
Phase
Low
Frequency
High
Frequency
Frequency (cyc/deg) Stimulus
When local phase across entire frequency
channels cooperate, then local phase
information dominates local magnitude
information.
Original Images
22
Result : Combination 12
Local
Magnitude
Local
Phase
Low
Frequency
23 of 51
High
Frequency
Original Images
Frequency (cyc/deg)
Result : Combination 12
Local
Magnitude
Local
Phase
Low
Frequency
24 of 51
High
Frequency
Original Images
Stimulus
Frequency (cyc/deg)
Result : Combination 12
48%
Local
Magnitude
52%
Local
Phase
Low
Frequency
25 of 51
High
Frequency
Frequency (cyc/deg) Stimulus
Original Images
Result : Combination 12
48%
Local
Magnitude
52%
Local
Phase
Low
Frequency
High
Frequency
Frequency (cyc/deg) Stimulus
Local phase and local magnitude have equal
importance for the appearance of an image
26 of 51
Original Images
Result : Combination 13
Local
Magnitude
Local
Phase
Low
Frequency
27 of 51
High
Frequency
Original Images
Frequency (cyc/deg)
Result : Combination 13
Local
Magnitude
Local
Phase
Low
Frequency
28 of 51
High
Frequency
Original Images
Stimulus
Frequency (cyc/deg)
Result : Combination 13
76%
Local
Magnitude
24%
Local
Phase
Low
Frequency
29 of 51
High
Frequency
Frequency (cyc/deg) Stimulus
Original Images
Result : Combination 13
76%
Local
Magnitude
24%
Local
Phase
Low
Frequency
High
Frequency
Frequency (cyc/deg) Stimulus
High frequency information is more important
than low frequency for image appearance.
30 of 51
Original Images
Discussion

Do we infer global phase from local phase?
31 of 51
Result : Combination 12
48%
Local
Magnitude
52%
Local
Phase
Low
Frequency
32 of 51
High
Frequency
Frequency (cyc/deg) Stimulus
Original Images
Discussion


Is there visual summation of local phase across frequency
channels?
33 of 51
Result : Combination 3
39%
Local
Magnitude
6%
55%
Local
Phase
Low
Frequency
34 of 51
High
Frequency
Frequency (cyc/deg) Stimulus
Original Images
Discussion



Why HVS needs local phase information?
35 of 51
Discussion




How local phase is computed in HVS?
36 of 51
Discussion





What is the neural basis for this computation?
37 of 51
Discussion

What is more important low frequency or high
frequency?
38 of 51
Result : Combination 13
76%
Local
Magnitude
24%
Local
Phase
Low
Frequency
39 of 51
High
Frequency
Frequency (cyc/deg) Stimulus
Original Images
Discussion

What is more important low frequency or high
frequency?
Previous research:


Low and high convey independent information about image
structure
For categorization task
1.
2.

Between class – Low frequency
Within class – High frequency
Information content in low and high frequency. Is this task
dependent?
40 of 51
Algorithm LMPD
Local Magnitude and Phase Distortion Rater

Algorithm was developed to rate the quality of local
phase distorted images using experimental results.

Algorithm computes local phase as well as local
magnitude distortions in an image.
41 of 51
Algorithm LMPD
Local Magnitude and Phase Distortion Rater

Local Magnitude distortion
Decompose original and distorted images in five scale and
ten orientation log-Gabor subbands.
For each scale
1.
2.
a.
b.
c.
d.
e.
42 of 51
Compute local energy maps of the original and distorted images.
Compute local MSE map between local energy maps of the original
and distorted images. Use block size of 16 × 16 for local MSE.
Collapse local MSE map via the L2 − norm into a single scalar value.
Compute correlation between local energy maps of original and
distorted images.
Compute local magnitude distortion Si(where, i is the index for
current scale) by multiplying scalar values obtained in step (c) and
(d).
Algorithm LMPD
Local Magnitude and Phase Distortion Rater
3.
Using Equation combine the local magnitude distortion
obtained for each scale in step (e) to compute local
magnitude distortion rating in the distorted image.
localmagnitudeDistortionRating = (𝑠1𝑝1 ) + (𝑠2𝑝2 ) + (𝑠3𝑝3 ) + (𝑠4𝑝4 )
+ (𝑠5𝑝5 )
𝑝1, 𝑝2, 𝑝3, 𝑝4, 𝑝5 are power coefficients with values of 4, 4, 2, 1.5
and 0.143 respectively
43 of 51
Algorithm LMPD
Local Magnitude and Phase Distortion Rater

Local phase distortion
1. Decompose original and distorted images in four levels
and four orientation complex wavelet subbands.
2. For each level of the complex wavelet subband
a.
b.
44 of 51
Extract local phase information of the original and distorted
image.
Compute local phase distortion E (where, i is the index for
current level) by computing MSE between local phase of the
original and distorted image obtained in step (a).
i
Algorithm LMPD
Local Magnitude and Phase Distortion Rater
3.
Using Equation combine the local Phase distortion
obtained for each scale in step (b) to compute local
phase distortion rating in the distorted image.
localphaseDistortionRating = (𝐸1𝑝1 ) + (𝐸2𝑝2 ) + (𝐸3𝑝3 ) + (𝐸4𝑝4 )
𝑝1, 𝑝2, 𝑝3, 𝑝4are power coefficients with values of 2.1, 2.4, 2.3,
and 2.2 respectively
45 of 51
Algorithm LMPD
Local Magnitude and Phase Distortion Rater

Final quality rating of distorted image
𝐿𝑀𝑃𝐷
= localmagnitudeDistortionRatingα
× localphaseDistortionRating(1−α)
α = 0.6
46 of 51
Algorithm LMPD
Local Magnitude and Phase Distortion Rater



Database of 48 phase distorted images.
Five subjects rated the distorted images.
Performance of LMPD compared with other image quality
assessment algorithms.
47 of 51
Algorithm LMPD
Local Magnitude and Phase Distortion Rater



Database of 48 phase distorted images.
Five subjects rated the distorted images.
Performance of LMPD compared with other image quality
assessment algorithms.
Correlation
48 of 51
PSNR
SSIM
CWSSI
M
NQM
VIF
MAD
LMPD
0.387
0.531
0.458
0.424
0.580
0.281
0.708
Conclusion

Local magnitude is most important information for
image appearance.

Local phase also has a substantial, sometimes
dominating contribution.

Local phase distortion of the images also has a
substantial effect on image quality.
49 of 51
Conclusion



We believe that an explicit mechanism
does exist in visual cortex for the
computation of local phase information.
50 of 51
Thank You

Questions?
For more stimuli examples please visit
http://vision.okstate.edu/localphase/
51 of 51