Fourier Analysis of Stochastic Sampling
For Assessing Bias and Variance in Integration
Kartic Subr, Jan Kautz
University College London
great sampling papers
Spectral analysis of
sampling must be
IMPORTANT!
BUT WHY?
numerical integration, you must try
assessing quality: eg. rendering
Shiny ball in motion
Shiny ball, out of focus
pixel
…
multi-dim integral
variance and bias
High variance
High bias
bias and variance
predict as a function of
sampling strategy and
integrand
High variance
High bias
variance-bias trade-off
High variance
High bias
analysis is non-trivial
Abstracting away the application…
0
numerical integration implies sampling
(N samples)
sampled integrand
0
numerical integration implies sampling
sampled integrand
0
the sampling function
integrand
sampling function
sampled integrand
multiply
sampling func. decides integration quality
integrand
sampling function
sampled function
multiply
strategies to improve estimators
1. modify weights
eg. quadrature rules
strategies to improve estimators
1. modify weights
eg. quadrature rules
2. modify locations
eg. importance sampling
abstract away strategy: use Fourier domain
1. modify weights
2. modify locations
eg. quadratureanalyse
rules sampling function in Fourier domain
abstract away strategy: use Fourier domain
1. modify weights
2. modify locations
sampling function in the Fourier domain
amplitude (sampling spectrum)
eg. quadrature rules
a. Distribution eg. importance sampling)
frequency
phase (sampling spectrum)
stochastic sampling & instances of spectra
Instances of sampling functions
Sampler
(Strategy 1)
draw
Instances of sampling spectra
Fourier
transform
assessing estimators using sampling spectra
Instances of sampling functions
Instances of sampling spectra
Sampler
(Strategy 1)
Which strategy is better? Metric?
Sampler
(Strategy 2)
frequency
accuracy (bias) and precision (variance)
Estimator 2 has lower bias but higher variance
reference
Estimator 1
Estimator 2
estimated value (bins)
overview
related work
Monte Carlo
rendering
Monte Carlo
sampling
signal processing
assessing sampling patterns
spectral analysis
of integration
stochastic jitter: undesirable but unavoidable
Jitter [Balakrishnan1962]
Point processes [Bartlett 1964]
signal processing
Impulse processes [Leneman 1966]
Shot noise [Bremaud et al. 2003]
we assess based on estimator bias and variance
Point statistics [Ripley 1977]
Frequency analysis [Dippe&Wold 85, Cook 86, Mitchell 91]
Discrepancy [Shirley 91]
Statistical hypotheses [Subr&Arvo 2007]
Others [Wei&Wang 11,Oztireli&Gross 12]
assessing sampling patterns
recent and most relevant
spectral analysis
of integration
numerical integration schemes [Luchini 1994; Durand 2011]
errors in visibility integration [Ramamoorthi et al. 12]
recent and most relevant
1.
2.
we derive estimator bias and variance in closed form
we consider sampling spectrum’s phase
spectral analysis
of integration
numerical integration schemes [Luchini 1994; Durand 2011]
errors in visibility integration [Ramamoorthi et al. 12]
Intuition
(now)
Formalism
(paper)
sampling function = sum of Dirac deltas
+
+
+
Review: in the Fourier domain …
Dirac delta
Fourier transform
Frequency
Imaginary
amplitude
phase
Real
primal
Fourier
Complex plane
Review: in the Fourier domain …
Dirac delta
Fourier transform
Frequency
Imaginary
Imaginary
Real
Real
primal
Fourier
Complex
plane
Complex
plane
amplitude spectrum is not flat
Fourier transform
=
=
+
+
+
+
+
+
primal
Fourier
sample contributions at a given frequency
1 2 3 4
5
At a given frequency
sampling function
Imaginary
3
5
Real
1
4
2
Complex plane
the sampling spectrum at a given frequency
sampling spectrum
Complex plane
given frequency
3
centroid
5
1
4
2
the sampling spectrum at a given frequency
sampling spectrum instances
given frequency
expected centroid
centroid variance
expected sampling spectrum and variance
expected amplitude of sampling spectrum
variance of sampling spectrum
DC
frequency
intuition: sampling spectrum’s phase is key
• without it, expected amplitude = 1!
– for unweighted samples, regardless of distribution
• cannot expect to know integrand’s phase
– amplitude + phase implies we know integrand!
Theoretical results
Result 1: estimator bias
sampling spectrum
bias
S
integrand’s spectrum
f
Implications
reference
frequency variable
1. S non zero only at 0 freq. (pure DC) => unbiased estimator
inner product
2. <S> complementary to f keeps bias low
3. What about phase?
expanded expression for bias
bias
expanded expression for bias
amplitude
bias
S
reference
f
S
f
phase
omitting phase for conservative bias prediction
amplitude
bias
S
reference
f
S
f
phase
new measure: ampl of expected sampling spectrum
ours
periodogram
Result 2: estimator variance
sampling spectrum
variance
S
integrand’s power spectrum
2
|| f ||
inner product
frequency variable
the equations say …
• Keep energy low at frequencies in sampling spectrum
– Where integrand has high energy
case study: Gaussian jittered sampling
1D Gaussian jitter
jitter using iid
Gaussian distributed
1D random variables
samples
1D Gaussian jitter in the Fourier domain
Fourier transformed
samples at an arbitrary
frequency
Jitter in position
manifests as
phase jitter
Imaginary
Complex plane
centroid
real
derived Gaussian jitter properties
• any starting configuration
• does not introduce bias
• variance-bias tradeoff
Testing integration using Gaussian jitter
random points
binary function
p/w constant function
p/w linear function
bias-variance trade-off using Gaussian jitter
variance
random
Gaussian jitter
low-discrepancy
Box jitter Poisson disk
grid
bias
log-variance
Gaussian jitter converges rapidly
Random:
Slope = -1
O(1/N)
Poisson disk
low-discrepancy
Box jitter
Gaussian jitter
Log-number of primary estimates
Conclusion: Studied sampling spectrum
sampling function
sampling
spectrum
integrand
integrand
spectrum
Conclusion: bias
bias depends on E(
sampling function
sampling
spectrum
integrand
integrand
spectrum
).
Conclusion: variance
bias depends on E(
sampling function
).
sampling
spectrum
integrand
integrand
spectrum
variance is V(
).
2
Acknowledgements
Take-home messages
3
5
1
4
No energy in sampling spectrum
at frequencies where
integrand has high energy
2
relative phase is key
Ideal sampling spectrum
Questions?
http://www.wordle.net/show/wrdl/6890169/FMCSIG13
Sorry, what? Handling finite domain?
• Integrand = integrand * box
conclusion