Analysis on errors due to photon noise and quantization
process with multiple images
The MIT Faculty has made this article openly available. Please share
how this access benefits you. Your story matters.
Citation
Li, Fulu et al. “Analysis on Errors Due to Photon Noise and
Quantization Process with Multiple Images.” 2010 44th Annual
Conference on Information Sciences and Systems (CISS).
Princeton, NJ, USA, 2010. 1-6. Copyright © 2010, IEEE
As Published
http://dx.doi.org/10.1109/CISS.2010.5464908
Publisher
Institute of Electrical and Electronics Engineers
Version
Final published version
Accessed
Thu May 26 19:16:10 EDT 2016
Citable Link
http://hdl.handle.net/1721.1/65842
Terms of Use
Article is made available in accordance with the publisher's policy
and may be subject to US copyright law. Please refer to the
publisher's site for terms of use.
Detailed Terms
Analysis on Errors due to Photon Noise and Quantization Process
with Multiple Images
Fulu Li, James Barabas, Ankit Mohan, Ramesh Raskar
The Media Laboratory, MIT, Cambridge, MA USA 02139
Email: fulu@media.mit.edu
Abstract.
In the scenes with deflective and/or
reflective medium such as fogs or mirrors, where the
contrast levels among the intensity values of the pixels in
the image are very small, even a small estimation
improvement on the intensity value of each pixel could
make a big difference for the perceptual quality of the
final image if we want to subtract the interference
components from the original image. In this paper we
analyze the average quantization error, total error and
photon noise from multiple images with fixed and varied
exposure time. We observe that by properly controlling
and varying exposure time with multiple images, one can
obtain an image with better performance than that of the
conventional wisdom, where multiple images are taken
repeatedly with the same exposure time.
Keywords: Photon Noise, Quantization Error, Images;
1. Introduction
Image noise is an inherent and integral part of the digital
imaging. In order to achieve the best performance from
digital imaging, it is essential to understand the impact of
noise, in particular the signal-dependent photon noise
[18], on the estimation of the intensity values of each
pixel in the image.
The motivation of this work is two-folds. First, for the
scenarios in which the contrast levels among the
intensity values of each pixel in the image are very small,
even a small improvement for the estimation of the
intensity values of the pixels in the image could
significantly improve the perceptual quality of the final
image. In other scenarios with reflective scenes by a
mirror or with the masking process in computational
photography, the contrast levels among the intensity
values of each pixel in the resulted image are also very
small. In this paper, we examine the tradeoffs to change
or not to change exposure time in order to achieve the
best performance from multiple images by collectively
reducing the errors due to photon noise and quantization
process, which will lead to a better resulted image.
Second, the availability of the "burst" mode for
commodity cameras provides new opportunities to get a
better image from multiple images by taking many
978-1-4244-7417-2/10/$26.00 ©201 0 IEEE
images quickly. A conventional wisdom to reduce
random noise is to take the average of multiple images
[I2] by taking many images of the same scene repeatedly
with the same exposure time. A natural question is that is
this conventional wisdom the best way to go in order to
get a better image from multiple images. In this paper,
we tend to answer the above question by analyzing the
trends of average quantization error, total error and
photon noise with fixed and varied exposure time to take
multiple images of the same scene.
The main results of this paper are as follows:
(a)
Quantization error can not be overcome when
the noise is zero with fixed exposure time.
(b)
Quantization error can be reduced when the
noise is zero and the exposure time is varied among
different images.
(c)
Quantization error can be reduced when the
noise isfinite and the exposure time is fixed among
different images.
(d)
Quantization error can be further reduced with
high probability when the noise is finite and the
exposure time is varied among different images
compared with that of the fixed exposure time for
multiple images.
The rest of the paper is organized as follows: we discuss
the related work in Section 2; we present preliminaries
regarding the models in Section 3; we analyze average
quantization errors, total errors and photon noise with
multiple images in Section 4; the conclusions are
presented in Section 5.
2. Related Work
Many researches have explored various ways to deal
with image noise and the lack of quantization resolution
in the face of small contrast levels in the image caused
by interference components such as fogs. One natural
way is to increase the number of quantization levels,
which could lead to higher cost to produce the camera
itself and it is also not trivial to do.
In [I2], Martinec conducted a systematic study on the
sources of noise in digital imaging and presented some
measurement results on image noise. The author also
discussed some aspects of noise reduction on raw data.
In [12], Martinec also mentioned the conventional
wisdom of repeatedly taking multiple images with the
same exposure time to reduce random noise.
In [4], Grossberg and Nayar presented an algorithm to
combine a few exposures to achieve high dynamic range.
The authors proved that simple summation combines all
the information in the individual exposures without loss.
The approach in [4] to find the optimal exposures for a
desired camera response function and a desired dynamic
range is to minimize a function of the difference between
the derivative of the camera response function and the
derivative of the desired camera response function plus a
term, which penalizes exposures that increase the noise
variance. As will be seen in our analysis that a larger
noise variance may not mean a larger quantization error
when combining multiple images.
Schechner et al in [20] proposed techniques for instant
dehazing using polarization to remove the effects of haze
from images. The method presented in [20] uses multiple
images taken through a polarizer at different orientations.
Recent work by Fattal in [3] presented a new method of
dehazing from single image by estimating the optical
transmission in hazy scenes given a single input image.
In [18], Ratner and Schechner presented a thorough and
insightful treatment of noise mechanisms in digital
imaging. The authors in [5,6,10,13] explored different
ways for noise estimation based on noise distributions
and camera response functions.
3. Preliminaries
In this section, we give a brief overview on the noise
model, quantization error model, and the related
theorems governing the corresponding statistical/random
processes.
As described in [12,13,18], image noise can be divided
into two categories: signal-dependent noise and signalindependent noise. As the name suggests, signalindependent noise, independent of the photon flux, is due
to dark current, amplifier noise and the quantizer in the
camera circuitry [18]. On the other hand, signaldependent noise is determined by the photon flux and the
uncertainty of the electron-photon conversion process
that occurs in the camera detector [18].
In this study, we are mostly interested in examining the
impact of photon shot noise, which is signal-dependent.
It is well known in this literature that the stream of
photons follow a Poisson distribution [6,12,13,18]. An
important characteristic of the Poisson distribution is that
the standard deviation of the fluctuations is equal to the
square root of the average count itself The fluctuations
in photon counts are perceived in images as noise, also
known as Poisson noise or photon shot noise [12].
As discussed in [6] that if the mean of a Poisson
distribution is relatively large, we can approximate a
Poisson distribution as a Gaussian distribution for
photon noise. However, for very dark pixels, the
Gaussian approximation of image noise may not be that
accurate.
3.3 The Quantization Error Model
Quantization error is due to digitization in the imaging
process. There are two kinds of quantization errors: the
approximation error and the clipping error. The
approximation error is caused by the "round -off' effect
of the analog signal due to the lack of quantization
resolution. The clipping error is due to inadequate
quantizer range limit. In our discussion, we mainly
consider the approximation error as the quantization
error. For a uniform quantizer, the quantization error is
bounded by the stepsize of the quantizer and we have
3.1 Camera Response Curve
The camera response curve is a curve showing the
relationship between the amount of incoming light,
which is often dictated by the length of exposure time,
and the image pixel intensity values of a digital camera.
In the following discussions, we assume linear camera
response in terms of the length of exposure time and we
have
veE) = a* E+ P
(1)
where E indicates the duration of the exposure time,
a and fJ are constant values. However, the principles
discussed here are equally applicable to other camera
response functions such as the logarithmic ones.
3.2 The Noise Model
-~/2~qe ~~/2
(2)
where q e denotes the quantization error and ~ is the
stepsize of the quantizer.
4. Quantization Error and Total Error with
Multiple Images
Assume that a random variable X, representing the
photon- flux signal, follows a probability distribution
with a mean of J1 and a variance of a 2 , i.e.,
2
We have N measurements of this random
process in the form of N images of the same scene with
the same exposure time, which are xl' X 2 ,... , X N .
X ""' (J1, (
) .
Notably, Xl' X2 ,... , X N are independent and identically
distributed (i.i.d.). As mentioned earlier, the stream of
photons, i.e., X, is Poisson distributed [6,12,18] and the
fluctuations in photon counts are considered as photon
shot noise [12]. As an important property of a Poisson
distribution, we have E{X} = Var(X) , i.e., f.1 = (j2 in
this case.
According to the Central Limit Theorem (CLT), we have
_
_ (j2
(3)
E{x} = f.1 and Var(x) =- ,
N
N
-'
LXi
where ~ = ..i..=!- .
N
By changing the exposure time, we have the
measurements of YI' Y2 ,...y N. Assuming linear camera
response function, we have
Yi=aix,l~i~N
(4)
where ai is a positive scalar to indicate the signal
changes caused by varied exposure time. In this new
setting, the normalized average x is defined as follows:
N
L~
for Xl , X2 ,..., XN, respectively.
the
Likewise,
let
quantized values
for
YI'Y2'···' YN, respectively.
value of aix, say aiE{x} , plus the signal-dependent
-
We need to emphasize that the term of s; in Eq. (9) is
different from the term of Si in Eq. (7) in that the shapes
of the Poisson/Gaussian noise distribution are quite
different in these two cases due to the fact that width of
the "bell-shape" noise distribution curve is proportional
to the square root of the real signal value.
Let Xte stand for the total error for Xl' X2 ,..., XN ,
meaning the difference between the average of the
quantized values of each individual measurement and the
real value of x , and we have
N
LXi,q
~te =1~-E{x}1
(10)
N
Likewise, let X te represent the total error for
YI' Y2 ,..., Y N' meaning the difference between the
the real signal value of x , i.e., E{x} , and we have
N
LYi,q
Xl' X2 ,..., XN ,meaning the average of the quantization
error at each measurement, and we have
-
x.«
N
I
=1
ai
i=l
N
- E{x} I
(11)
-
-
Notably, the difference between Xaqe in Eq. (6) and Xte
i=l
(6)
N
In this paper, we only consider signal-dependent photon
shot noise as most of the related work in this literature
does [6]. The signal-independent noise can not be
exploited for the reduction of quantization error and it
can be dealt separately. Notably, the measurement value
of Xi is the real signal value of x , say E{x} , plus the
Xaqe =
(9)
Yi =aiE{x}+s;
normalized average of the quantized values of Yi sand
Let xaqedenote the average quantization error (AQE) for
Llxi -Xi,q
(8)
Note that the measurement value of Yi is the real signal
i=l a i
x =-(5)
N
Let us further define XI,q' X2,q ,..., XN,q as the quantized
YI,q' Y2,q'· .. ,y N,q denote
v.:
Yi I
i=l
= ------
photon shot noise, say s; , which is the fluctuation of the
photon counts. So, we have
N
values
LI
X aqe
signal-dependent photon shot noise, say s i , which is the
fluctuation of photon counts. So, we have
(7)
Xi =E{X}+Si
-
Likewise, let Xaqe indicate the average quantization error
for YI, Y2 ,..., Y N' meaning the normalized average of
the quantization error at each measurement, and we have
in Eq. (10) is that x aqe is simply the average of the
-
quantization error at each measurement and x» is the
difference between the average of the quantized value at
each measurement and the real signal value of x, i.e.,
E{x}. Similarly, the same relationship applies to Xaqe
-
and Xte.
Regarding the quantization error with multiple
measurements (multiple images in our case), we have the
following observations:
Lemma 1: Quantization error can not be overcome
when the noise is zero with fixed exposure time.
To see this, let us have a look at the following
quantization example in Fig. 1. Without loss of
generality, let us assume that the signal value of x lands
between a and (a + 0.5), where a is an integer. The
quantization error in this example is a - x in this
example. If the noise is zero, the total error X te as
defined in Eq. (10) will remain the same as I a - x I no
matter how many times we try the measurements.
a
x
a+l
a+O.5
Figure 1: An illustration of quantization error without
noise.
Lemma 2: Quantization error can be reduced when the
noise is zero and the exposure time is varied among
different images.
Assume that we have a linear response camera and the
intensity value of a pixel is determined linearly by the
duration of the exposure time when the image is taken.
Without loss of generality, let us have a look at the
examples illustrated in Fig. 2. In Fig. 2.(a), the signal
value Yi will be truncated as a (Yi ' s closest integer
point), which leads to the quantization error of (a - Y i)' a
negative value. On the other hand, if we change the
exposure time and the corresponding signal value of Y j
could land in a different region from that of Y i 'S. By
properly controlling the exposure time, it is possible that
Y j lands in a region between (b +0.5) and (b +l),
where b is an integer. As a result, the quantized value of
Y j would be (b + 1) , which leads to the quantization error
of (b + 1- Y j ), a positive value. Therefore, the total
error X te as defined in Eq. (11) can be reduced by the
"cancelling-out" effect of " overestimates" (see Fig. 2.(b)
)and "underestimates"
(see Fig. 2.(a» if we take
multiple images with controlled and varied exposure
time.
Yi
Without loss of generality, let us look at the following
example depicted in Fig. 3, where we have a real signal
value of s and the photon shot noise, which oscillates
around the signal value of s with a "bell-shape"
distribution in terms of its density. The horizontal axis
indicates possible measurement values and the vertical
axis denotes the probability density function of the
corresponding photon shot noise distribution. Let
n denote the random variable of photon noise
( n E [A - s, D - s] in this example). As we can see that
without noise, the quantization error will be (a - s) ,
where a is s 's closest integer and (a - s) is a negative
value in this case. With noise, if the value of the signal
plus noise ( s + n i ) lands within the range of [A , C],
where C is equal to a + 0.5, the quantized value of
s + n i is still the value of a, which leads to an
underestimate of the real value of s. However, if the
value of signal plus noise of (s + n j ) (the noise term of
n j varies among different images for the same pixel)
lands within the range of [C, D], then the quantized
value of (s + n j ) becomes a + 1, which leads to an
overestimate of the real value of s. If we repeat the
measurements for multiple times from multiple images
with the fixed exposure time, the total error as defined in
Eq. (10) from multiple measurements can be reduced due
to the "cancelling-out" effect of "overestimates" and
"underestimates".
a+l
a+O.5
(a) signal
Lemma 3: Quantization error can be reduced when the
noise is finite and the exposure time is fixed among
different images.
is obtained with the exposure time of
ei
seconds.
s
A
a- O.s
b
(b) signal
b+O.5
v,
a
C
a+ O.5
a+ l
b+l
Y j is obtained with the exposure time of e j
seconds.
Figure 2: An illustration of quantization error with
varied exposure time.
Figure 3: An illustration of quantization error with
photon shot noise.
Lemma 4: For the same limited number of images (N) ,
the total error can be further reduced with high
probability when the noise is finite and the exposure time
is varied properly among different image s even though
the photon noise may not be further reduced compared
with multiple images with the same exposure time. More
formally, by properly choosing varied exposure time,
-
-'
with high probability we can have Xte ~ Xte for the same
limited number of images (N) even tough the photon
noise may not be further reduced compared with the case
with
fixed
exposure
time,
i.e.,
we
have
-
-
Var(x ) ~ Var(x).
E{y;} = E{a;x}
Xte ~
==
a i f.1 ,
(12)
where a; is a positive constant and 1 ~ i ~ N .
Var(y;) = Var(a;x)
where 1 ~ i ~ N .
==
a; x Var(x)
==
a;a- 2 ,
(13)
We further have
First, let us look at the first part of the lemma stating that
by properly choosing varied exposure time, we can have
-
a; x E{x}
==
-
x«. Recall the definition of
-
in Eq. (10) that the
total error for the case with fixed exposure time is simply
the absolute value of the difference between the average
of the quantized values (xi,qs) at each measurement and
Xte
N
E{~
N
LaixE{x}
i=l
LE{Yi}
} == _i=_l_ _
N
La i
i=l
the real value of the signal x. Likewise, the definition of
in Eq. (11) states that the total error for the case with
varied exposure time is the absolute value of the
difference between the average of the normalized
(14)
Xte
quantized values (Y;,q s) at each measurement and the
a;
;=1
;=1
We further have
N
real value of the signal x. Clearly, by properly choosing
a sequence of varied exposure time, with high probability
,
LYi
Var(~ ) == Var( i~l
)=
the normalized quantized values of Y;,q s could provide
(La i)2
i~1
La i
;=1
i=l
Note that Y1' Y2 ,..., Y N are independent measurements.
a;
richer granularities for the "cancelling-out" effects of
"overestimates" and "underestimates" of the real value of
the signal x, compared with the granularity of Xi,q s,
which is the quantization unit. Further, as we vary the
exposure time, the signal-dependent photon noise also
varies accordingly in the sense that the "bell shape" (see
illustrations in Fig. 3) of the photon noise
By properly choosing varied exposure time, the chances
of the "cancelling-out" effect of "overestimates" and
"underestimates" could be improved due to more diverse
distributions of signal values of Y; s and the
N
Var(LYi) (15)
N
From (15), we have
N
N
La; xVar(x)
i=l
LVar(Yi)
Var(~ ) == _i=_l_
N-_a i )2
i=l
(L
N
2xLa;
ai=l
corresponding signal-dependent noises. Notably, x; s are
(16)
independent and identically distributed , while Y; s are
independent but no longer identically distributed.
Therefore, by properly choosing varied exposure time,
with high probability we can have
-'
Xte ~ Xte .
Now let us prove the second part of the lemma stating
that the photon noise of
-
Var(x) may not be further
reduced compared with Var(x) , which makes the results
of this lemma even more interesting and counterintuitive.
To see this, we have
Note that aI' a 2 ,..., aN are positive numbers. If and only
if a1 == a2 == ... == aN, then we have
-'
a-2
Var(x ) == Var(x) == N
(17)
Otherwise, we have
-
-
Var(x ) > Var(x)
(18)
As we can see that the expectation of
~ 'remains the
same as that of ~ but the variance (the photon noise)
grows, i.e., Var(x ) ~ Var(x).
What this lemma tells us is that photon noise is not
necessarily correlated with the trend of the total error as
defined in Eq. (10) or Eq. (11).
5. Conclusion
In this paper we present an indepth analysis on average
quantization error, total error and photon noise from
multiple images of the same scene with fixed or varied
exposure time. In the scenarios with heavy fogs or
reflective scenes where the contrast levels among the
intensity values of the pixels in an image is very small, a
small improvement on the accuracy of the estimated
pixel value could make a big difference for the final
perceptual quality of the image if we want to subtract the
interference component from the original image. We
examine the reduction of total error with multiple images
of the same scene with fixed or varied exposure time.
Reference
[1] CHAVEZ, P. S., "An improved dark-object
subtraction technique for atmonspheric scattering
correction of multispectral data", Remote Sensing of
Environment 24, 450--479, 1988.
[2] DEBEVEC, P. E., AND MALIK, 1., "Recovering
high dynamic range radiance maps from photographs",
in ACM SIGGRAPH 1997,369-378.
[3] FATTAL , R., "Single Image Dehazing", in Proc. of
ACM Siggraph '2008.
[4] GROSSBERG, M., S. K. NAYAR, "High Dynamic
Range from Multiple Images: Which Exposures to
Combine?", in IEEE Workshop CPMCV, 2003.
[5] HEALEY G. E., R. KONDEPUDY, "Radiometric
ccd camera calibration and noise estimation", in IEEE
Trans. on PAMI, 1994.
[6] HUANG Y., 1. KIM, I. KWEON, "Sensor noise
modeling using the Skellam distribution: Application to
the color edge detection", in Proc. of IEEE CVPR '2007.
[7] HYVRINEN , A., AND OJA, E., "Independent
component analysis: algorithms and applications",
Neural Networks 13, 411- 430, 2000.
[8] KAMGAR-PARSI 8. , 8. KAMGAR-PARSI ,
"Evaluation of Quantization Error in Computer Vision",
in IEEE Trans. on PAM/, 1989.
[9] LARSON, G. W., RUSHMEIER, H., AND PIATKO,
c., "A visibility matching tone reproduction operator for
high dynamic range scenes", IEEE Trans on
Visualization and Computer Graphics 3, 4, 291-306,
1997.
[10] LIU, c., W. T. FREEMAN, R. SZELISKI and S. 8.
KANG, "Noise estimation from a single image", in Proc.
ofIEEE CVPR '2006.
[I I] LU, 1., AND D. M. Healy "Contrast enhancement
via multi scale gradient transformation",
in IEEE
International Conference on Image Processing, 482--486,
1994.
[12] MARTINEC, E., "Noise, Dynamic Range and Bit
Depth in Digital SLRs", http://theory.uchicago.edu/
ejm/pix/20d/tests/noise/index .html.
[13] MATSUSHITA, Y., S. LIN, "A Probabilistic
Intensity Similarity Measure based on Noise
Distribution", in Proc. ofIEEE CVPR '2007.
[14] NARASIMHAN, S. G., AND NAY AR, S. K.,
"Chromatic framework for vision in bad weather", in
Proceedings of IEEE CVPR, 598-605, 2000 .
[15] NAYAR, S. K., AND NARASIMHAN, S. G.,
"Vision in bad weather", in Proceedings of IEEE CVPR,
820, 1999.
[16] OAKLEY, 1. P., AND BU, H., "Correctio~ of
simple contrast loss in color images", IEEE Transactions
on Image Processing 16,2,511-522,2007.
[17] RAHMAN, Z., JOBSON, D., AND WOODELL,
G., "Multiscale retinex for color image enhancement", in
Proc. of IEEE ICIP, 1996.
[18] RATNER, N., Y. SCHECHNER, "Illumination
Multiplexing within Fundamental Limits", in Proc. of
IEEE CVPR '2007.
[19] ROSSUM, M. V., AND NIEUWENHUIZEN, T.,
"Multiple scattering of classical waves: microscopy,
mesoscopy and diffusion", Reviews of Modern Physics,
vol. 71,313-371,1999.
[20] SCHECHNER, Y., S. NARASIMHAN, S. K.
NAYAR, "Instant Dehazing of Images Using
Polarization", in Proc. oflEE C VPR '200 I.
[21] SHWARTZ, S., NAMER, E., Y ,
AND
SCHECHNER, Y., "Blind haze separation",
in
Proceedings ofIEEE CVPR, 1984-1991 , 2006.
[22] SINGH, M., AND ANDERSON, 8. , "Toward a
perceptual theory of transparency", Psychological
Review, No. 109,492-519,2002.
[23] TAN , K., AND OAKLEY , 1. P., "Enhancement of
color images in poor visibility conditions", in ICIP 2000.
[24] TAN, R. T., "Visibility in bad weather from a single
image", Proceedings ofIEEE CVPR, 2008.
[25] YUAN, L., SUN, 1., QUAN , L., AND SHUM, H.Y., "Image deblurring with blurred/noisy image pairs",
ACM Transactions on Graphics 26, 3, 1,2007.
[26] ZHANG, Y., GUINDON, 8., AND CIHLAR, 1.,
"An image transform to characterize and compensate for
spatial variations in thin cloud contamination of landsat
images", Remote Sensing of Environment 82 (October),
173-187, 2002.