INFORMATIK
Design of a Tone Mapping Operator
for High Dynamic Range Images based
upon Psychophysical Evaluation and
Preference Mapping
F. Drago1, W. Martens2, K. Myszkowski3,
and N. Chiba1
1Iwate
University and 2Aizu University, Japan
3Max-Planck-Institut für Informatik, Germany
Overview
•
•
•
•
•
Motivation
Previous work
Psychophysical experiment
Enhancements of Retinex for HDR images
Conclusions
INFORMATIK
Motivation
Many applications
• Lighting simulation and realistic rendering
• High Dynamic Range photography
• Multimedia: distributing HDR video streams
INFORMATIK
HDR Photographs + Rendering:
Real World Lighting
INFORMATIK
1) Photographs of mirror sphere at varying exposure times
3) Use as light source in Monte
Carlo radiosity algorithm
2) High-dynamic
range environment map
Philippe Bekaert
Goals
INFORMATIK
• Technical requirement
– Match the dynamic range of image to the range
available on a given display device
• Various objectives
– Get good perceptual match between the real-world
and corresponding images
– Reproducing details
– Maximize reproducible contrast
– Just to get “nice-looking” images
Various Classifications
• Theoretical foundations
– Perception-based
– Pure image processing techniques
• Mapping function
– Global – the same for all pixels
– Local – depends on local image contents
• Temporal processing
– Static
– Dynamic
INFORMATIK
Previous Work:
Global Methods
INFORMATIK
Perception-based
• Tumblin and Rushmeier (1993,1999)
– Brightness matching
• Ward (1994), Ferwerda et al. (1996)
– Contrast matching (a linear function is used)
• Ward et al. (1997)
– Adjusting image histogram to avoid exceeding
display contrast in respect to the real-world scene
Efficiency-driven
• Schlick (1994)
– Rational functions
Examples
Ferwerda et al.
Tumblin (1999)
INFORMATIK
Ward et al.
Schlick
Previous Work:
Local Methods
INFORMATIK
• Early methods – prone to halo artifacts
– Chiu et al. (1993), Schlick (1994),
– Land (1971), Jobson et al. (1997): Retinex
– Pattanaik et al. (1998): The most comprehensive
model of HVS used in CG
• LCIS: Tumblin and Turk (1999)
– Based on an anisotropic diffusion procedure
– Emphasize on details but compress excessively
contrast
• New wave: Fattal et al., Reinhard et al., Durand and
Dorsey, Ashikhmin (2002)
Examples
Tumblin and Turk
INFORMATIK
Ashikhmin
Retinex
Examples
Durand and Dorsey
INFORMATIK
Fattal et al.
Reinhard et al.
INFORMATIK
Ashikhmin
Durand and Dorsey
Fattal et al.
Reinhard et al.
Psychophysical Experiment
INFORMATIK
• Perceptual evaluation of subject preference by pairwise
comparison of tone mapped images
• Seven tone mapping algorithms examined:
– Tumblin and Rushmeier (1993),
– Ferwerda et al. (1996),
– Ward et al. (1997),
– Schlick (1994),
– Retinex - based on Funt and Ciurea (2001) implementation
but with our extensions toward suppressing halo
– Reinhard et al. (2002) – photographic method
– Tumblin and Turk (1999) - LCIS
• Four scenes considered
INFORMATIK
Statistical Data Processing
INFORMATIK
• 11 subjects participated
• Dissimilarity ratings for pairwise comparisons of
images submitted to Individual Differences Scaling
(INDSCAL) analysis
• Stimulus Space configures the stimuli such that
Euclidian distances between the stimuli match the
obtained dissimilarity judgments
• Axes labeled based upon correlation of the
dimensional coordinates with independently
generated attribute ratings (naturalness, detail and
contrast reproduction)
• “Ideal” preference point obtained through PREFMAP
analysis
Subject Preferences
INFORMATIK
–
–
–
–
–
–
T: Tumblin & R.
V: Ferwerda et al.
H: Ward et al.
Q: Schlick
X: Retinex
P: Reinhard et al.
Retinex
INFORMATIK
We use the “Frankle-McCann Retinex” algorithm
• ratio-product-reset-average
– NP(x,y) new pixel value is obtained from the original image
R() and previous iteration image OP() as follows:
(log OP ( xs , ys )  log R( x, y )  log( R( xs , ys ))  log OP ( x, y )
log NP ( x, y ) 
2
– Reset test
(logOP( xs, ys)  log R( x, y)  log(R( xs, ys))  log Lscene
max
• In each iteration (the number of iterations predefined by the user)
– the distance D between pixels (x,y) and (xs,ys) is halved
– the direction for pixel comparison is rotated 90o clockwise
Retinex Extensions: for HDR
INFORMATIK
• Main problem: Suppressing halo effects
– Adding counterclockwise rotation of the path
suggested by Coopers
– Spatially varying levels of pixel interaction based
contrast information
Suggested by Sobol, but we use a smooth
function for clipping
– Adjusting a reset ratio to the maximum luminance
of the display device instead of the maximum
luminance of the scene
Halo Reduction:
Retinex Rotation
Clockwise
CounterClockwise
All images for 40 iterations
INFORMATIK
Both Ways
Halo Reduction:
Retinex Contrast Crop with Bias
INFORMATIK
(log OP ( xs , ys )  log R( x, y )  log( R( xs , ys ))  log OP ( x, y )
log NP ( x, y ) 
2
Bias function f ( x)  x
log a
log 0.5
ContrastClip  D
log a
log 0.5
 0.7
Contrast  log R( x, y )  log R( xs, ys)
if (Contrast  ContrastClip)
Contrast  ContrastClip;
else if (Contrast  ContrastClip)
Contrast  ContrastClip;
Halo Reduction:
Retinex Contrast Crop with Bias
Standard Retinex
33 iterations cw and ccw
INFORMATIK
The same settings
but crop with bias added
Halo Reduction:
Retinex Contrast Crop with Bias
ContrastClip  D
a  0.8
33 Retinex iterations
log a
log 0.5
INFORMATIK
 0.7
a  0.95
33 Retinex iterations
Halo Reduction:
Retinex Contrast Crop with Bias
ContrastClip  D
log a
log 0.5
INFORMATIK
 0.7
a  0.95
4 Retinex iterations
30 Retinex iterations
Retinex Maximum Reset
INFORMATIK
(logOP( xs, ys)  log R( x, y)  log(R( xs, ys))  log Ldisplay
max
change from Lscene
max
to Ldisplay
max
Maximum = 226.5 cd/m^2
Maximum = 100 cd/m^2
INFORMATIK
Linear
mapping
Retinex
4 iterations
Extended
Retinex
4 iterations
Extended
Retinex
4 iterations
Lscene
max
Ldisplay
max
Retinex + Tone Mapping Op.
Ferwerda et al. (1996)
Logmap - new
INFORMATIK
Logmap Equation
INFORMATIK
Adaptive
Logarithmic
Mapping
Performance:
• Software
– 30 fps on
PentiumIV,
2.2GHz
• Hardware
–?
INFORMATIK
Conclusions
INFORMATIK
• We performed psychophysical of seven existing tone
mapping operators. More details in our TechRep:
http://data.mpi-sb.mpg.de/internet/reports.nsf/AG4NumberView?OpenView
• Good performance of Retinex in the experiment
encouraged us extend it toward reducing hallo
artifacts
• Addind a regular tone mapping processing atop of
Retinex results make the resulting images more
independent on the number of Retinex iterations and
improve the image naturalness
• Future work: repeating psychophysical with all recent
local tone mapping operators and our extended
Retinex
Color Balance Correction
Retinex Applied to All Channels
in LMS Color Space
INFORMATIK
Stanford Memorial Church
Photograph
INFORMATIK
Stanford Memorial Church
Photograph
INFORMATIK
Acknowledgments
INFORMATIK
We would like to thank Michael Ashikhmin,
Paul Debevec, Fredo Durand, Dani
Lischinski, Eric Reinhard, and Greg Ward for
providing us with some images used in this
presentation.
We would like also to thank Greg Ward for
his precious comments concerning our
work.