CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Image Processing & Antialiasing
Part II (Aliasing, Sampling, Convolution, and Filtering)
Andries van Dam©
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Outline










Overview
Example Applications
Jaggies & Aliasing
Sampling & Duals
Convolution
Filtering
Scaling
Reconstruction
Scaling, continued
Implementation
Andries van Dam©
10/1/2015
2/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Jaggies & Aliasing

“Jaggies” an informal name for artifacts
from poorly representing continuous
geometry by a discrete 2D grid of pixels – a
form of aliasing (later…)


Jaggies are a manifestation of sampling error
and loss of information (aliasing of high
frequency components by low frequency
ones)
Effect of jaggies can be reduced by antialiasing, which smoothes out the pixels
around the jaggies by averaging


Shades of gray instead of sharp black/white
transitions
Diminishes HVS’ “edge detection” response to
sharp transitions (Mach banding)
Andries van Dam©
10/1/2015
3/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Representing lines: Point sampling, single pixel

Midpoint algorithm: in each column, pick the pixel
with the closest center to the line

A form of point sampling: sample the line at each of the
integer X values

Pick a single pixel to represent the line’s intensity, full on
or full off

Doubling resolution in x and y only lessens the
problem, but costs 4 times memory, bandwidth, and
scan conversion time!

Note: This works for -1 < slope < 1, use rows instead of
columns for the other case or there will be gaps in the
line
Andries van Dam©
10/1/2015
Line approximation
using point sampling
Approximating same line
at 2x the resolution
4/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Representing lines: Area sampling

Represent the line as a unit width rectangle, use multiple pixels overlapping the rectangle
(for now we think of pixels as squares)

Instead of full on/off, calculate each pixel intensity proportional to the area covered by
the unit rectangle
A form of unweighted area sampling – stay tuned:




Only pixels covered by primitive can contribute
Distance of pixel center to line doesn’t matter
Typically have more than one pixel per column so can go gradually from dark for pixels
covered by the line to white background; the more area of overlap, the darker the pixel
Andries van Dam©
10/1/2015
5/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Filtering as form of averaging: filter is a weighting function


Apply a unweighted or weighted average to each sample and it’s neighbors
Example: filtering for student height or weight – give each sample equal
weight or diminish influence of samples further away
Pre-Filtered
samples

Weight function = centered at some
value of x, often a pixel center
Post-filtered samples (decreases
contrast between adjacent pixels): red
= added value, light blue is subtracted
area
Weight function gives weight for incremental area centered at
 The volume of a filter function needs to be 1 to preserve overall
brightness, so W =1 over its “support”, the area it covers, 0 elsewhere
Andries van Dam©
10/1/2015
6/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
“Box Filter” Represents Unweighted Area Sampling

Box filter is constant over all area and is a
single pixel wide here, but could vary in width

For each pixel intersecting the line, intensity
contributed by each sub-area of intersection
𝑑𝐴 is 𝑾 𝑥, 𝑦 ∗ 𝑑𝐴

𝑾
Then total intensity of the pixel (between 0
and 1) integrated over area of overlap is:
𝑾 𝑥, 𝑦 𝑑𝐴
𝐴

Integral is volume over area of overlap (in
above figure, a rectangular wedge)
Andries van Dam©
10/1/2015
7/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
“Cone Filter” for Weighted Area Sampling

Area sampling, but the overlap between filter
and primitive is weighted so that sub-areas
with 𝑑𝐴 closer to center of pixel count more

Cone has:


Linear falloff

Circular symmetry

Width of 2 (called support)
Intensity of pixel is the “subvolume” inside the
cone over the line (see picture)
Andries van Dam©
10/1/2015
8/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Weighted Area Sampling (Continued)
2-unit circular
support of filter
Pixel center (+)
Differential
area dA1
Differential
area dA2
Primitive
Andries van Dam©
W
Area of overlap
between support
and primitive
W(𝑥, 𝑦) is the weight which is multiplied
with 𝑑𝐴 at (𝑥, 𝑦); normalize W to make
volume under cone = 1
10/1/2015
9/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Another Look at Point Sampling – Even Box Filter is Better!
A
C
D
B
Point-sampling problems. Samples are shown as black dots. Object A and C are sampled,
but corresponding objects B and D are not.

This simplistic scan conversion algorithm only asks if a mathematical
point is inside the primitive or not

Bad for sub-pixel detail which is very common in high-quality rendering
where there may be many more micro-polygons than pixels!
Andries van Dam©
10/1/2015
10/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Another Look at Unweighted Area Sampling: Box filter

Box filter



Support: 1 pixel
Sets intensity proportional to area of overlap
Creates “winking” of adjacent pixels as a small triangle translates
(b)
Unweighted area sampling. (a) All sub-areas in the pixel are weighted
equally. (b) Changes in computed intensities as an object moves between
pixels.
Andries van Dam©
10/1/2015
11/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Another Look at Weighted Area Sampling: Pyramid filter

Pyramid filter


Support: 1 pixel
Approximates circular cone to emphasize area of overlap close to center of
pixel
(b)
Weighted area sampling. (a) sub-areas in the pixel are weighted differently
as a function of distance to the center of the pixel. (b) Changes in
computed intensities as an object moves between pixels.
Andries van Dam©
10/1/2015
12/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Another Look at Weighted Area Sampling: Cone filter

Cone filter


Support: 2 pixels
Greater smoothness in the changes of intensity
(b)
Weighted area sampling with overlap. (a) Typical weighting function. (b)
Changes in computed intensities as an object moves between pixels.
Andries van Dam©
10/1/2015
13/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Pseudocode and Results
for each sample point p: //p need not be integer!
place filter centered over p
for each pixel q under filter:
weight = filter value over q
p.intensity += weight * q.intensity
Aliased
Andries van Dam©
Anti-aliased
10/1/2015
Related phenomenon:
Moire patterns
“Crawlie Demo”
14/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Anti-Aliasing Example
Close-up of original, aliased render
Antialiasing Techniques:
Blur filter – weighted average of
neighboring pixels
Supersampling - sample multiple
points within a given pixel and
average the result
Checkerboard with Supersampling
Supersampling and Blurring
Andries van Dam©
10/1/2015
15/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Outline










Overview
Example Applications
Jaggies & Aliasing
Sampling & Duals
Convolution
Filtering
Scaling
Reconstruction
Scaling, continued
Implementation
Andries van Dam©
10/1/2015
16/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Aliasing and Anti-Aliasing: Intro to Signal Processing

Easier to discuss aliasing and anti-aliasing of images and image scaling than
of rendering anti-aliased geometric primitives - return at end of this unit

Need to do a brief intro to signal processing, since an image can be thought
of as a sequence of scan lines, each displaying continuous intensity signals
(photograph on fine-grain film) or discrete signals (pixels captured by a
digital camera)

For images, light waves vary in intensity in 2-space while sound waves vary
in time and pressure

Signals are complex waveforms but they can be decomposed into
combinations of simpler waveforms (sines and cosines) of varying
frequency, amplitude and phase: Fourier synthesis
Andries van Dam©
10/1/2015
17/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Sampling and Scaling of Images
(images courtesy of George Wolburg, Columbia University, in 2nd Edition Computer Graphics: P&P)


Scan converting an image is digitizing (sampling) a series of continuous
intensity functions, one per scan line.
We will use single scan lines for simplicity, but everything still applies to
full images
Scan line from synthetic scene
Andries van Dam©
10/1/2015
Scan line from natural scene
18/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
The Sampling/Reconstruction/Display Pipeline – overview
Original continuous signal:
𝑢: ℝ → ℝ
Sampled signal (lose information):
𝑆: ℤ → ℝ: 𝑛 ⟼ 𝑢 𝑛
Reconstructed signal (approximation
to original):
𝑆: ℝ → ℝ
(many reconstruction methods)
One reconstruction method (linear interpolation):
𝑆 𝑥 = 𝑛+1−𝑥 ∙𝑆 𝑛 + 𝑥−𝑛 ∙𝑆 𝑛+1 , 𝑛 = 𝑥
Andries van Dam©
10/1/2015
19/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Fourier Waveform Synthesis – Approximate a Continuous Signal


Any signal can be
approximated by
summing sine (and
cosine) waves of
different frequencies,
phases, and amplitudes
A signal has 2
representations. We’re
familiar with the
spatial domain, but
every signal also can be
represented in the
frequency domain
Andries van Dam©
10/1/2015
Approximation of scan
line signal from image
improves with addition of
more sine waves
20/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Digression: Music

A vibrating string creates a pressure wave with multiple frequency components

Intensity/volume of sound is related to amplitudes of components of pressure
wave

Lowest frequency is “fundamental frequency”

Integer multiples of the fundamental frequency are
called “harmonics” or “overtones” (non-integral
multiples also exist)
Power of two multiples of the fundamental
frequency are called “octaves”

Labels show 1/f -- the wavelength
For more, applets to interact with these concepts:
http://homepages.gac.edu/~huber/fourier/index.html
http://www.phy.ntnu.edu.tw/ntnujava/index.php?topic=17
http://www.chem.uoa.gr/Applets/AppletFourier/Appl_Fourier2.html
Andries van Dam©
10/1/2015
21/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Analogous Operations, Duals

Easier to visualize signals and filtering/averaging in the frequency domain

Many problems are easier solved by transforming to another problem (an untransforming!)


Characterize a signal by its frequency spectrum showing amplitudes of component waveforms

Signal and frequency domains are duals and represent identical information.

Easier to filter in frequency domain than in signal domain, as we shall see.
Take a familiar problem: multiplication of numbers


Can take logarithm of number, perform operations on log(number), then take antilogarithm)
Dual of multiplication is addition in logarithmic “dual”
If ab=c then
log(a) + log(b) = log(c)

This “invertible transformation” makes slide-rules such effective tools for multiplication: manipulating sliders
corresponds to manipulating numbers via their logs
Andries van Dam©
10/1/2015
22/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Frequency Spectrum of a Signal




Sine wave is characterized by amplitude
and frequency
Frequency of a sine wave is number of
cycles per second for audio, or number of
cycles per unit length (e.g., inter-pixel
distance) for image scan lines
Can characterize any waveform by
enumerating amplitude and frequency of all
its component sine waves (Fourier
transform – see chapter 18 in book)
This can be plotted as a “frequency
spectrum”, a.k.a. power spectrum, (we
ignore negative frequencies, but they are
needed for mathematical correctness)
Signal Domain
Frequency Domain
To see spatial and frequency domains of simple signals:
http://www.cs.brown.edu/exploratories/freeSoftware/reposit
ory/edu/brown/cs/exploratories/applets/fft1DApp/1d_fast_f
ourier_transform_guide.html
Andries van Dam©
10/1/2015
23/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Sine waves in 2D

Let white be max positive amplitude, black max negative amplitude
Low Frequency
Higher Frequency
(Harmonic – integer multiple)
http://cns-alumni.bu.edu/~slehar/fourier/fourier.html
Andries van Dam©
10/1/2015
Images: Hays
24/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Simple Fourier Synthesis example (1/2)

example : g(t) = sin(2πf t) + (1/3)sin(2π(3f) t)
+
Andries van Dam©
10/1/2015
=
Slides: Hays 25/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Simple Fourier Synthesis example (2/2)

Approximate signal using sines and cosines
Spatial domain
f(x)
x
Andries van Dam©
10/1/2015
Original
Partial
Approximation
Function
approximation
with
5 cosines
with 2
3 cosines
4
We are just
summing waves to
get bolded blue
function to
approximate the
red signal
26/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Sampling: The Nyquist Limit

In order to capture all frequencies in a signal, we must sample at a rate that is higher than 2 times the highest
frequency (with a non-zero value from the Fourier transformation) in the signal (the Nyquist limit).
Here is an approximate analog sine wave:

The sine wave sampled at an acceptable rate (4 times the highest frequency):

Reconstructed wave based on these samples:

Andries van Dam©
10/1/2015
27/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Aliasing: Know Thine Enemy

Aliasing occurs when we sample a signal at less than twice maximum frequency
Here is our analog sine wave again:

Here is the sine wave sampled at too low a rate:

Here is the reconstructed wave based on these samples:

The reconstruction isn’t even close!

Andries van Dam©
10/1/2015
28/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Temporal Aliasing: Another Sampling Error

Ever seen tires spin in a movie? Have you ever noticed that sometimes,
they seem to be spinning backwards?

It’s because the video frame-rate is lower than twice the frequency at which
the wheels spin. This is temporal aliasing!

You see this a lot in movies
because the effect is so striking.
It’s known as the stage-coach
effect.

Porsche Dyno Test
Andries van Dam©
10/1/2015
29/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Sampling at the Nyquist Limit

Sampling right at the Nyquist limit can also
be problematic

Here is our perfect analog sine wave again:

Here is the sine wave sampled at the Nyquist
limit. This time it works fine:

Here is the same sine wave sampled at the
Nyquist limit, with the sample points shifted.
Now we get no signal:

For an applet dedicated to illustrating the
Nyquist Limit:
Andries van Dam©
10/1/2015
http://www.cs.brown.edu/exploratories/freeSoftware/repositor
y/edu/brown/cs/exploratories/applets/nyquist/nyquist_limit_gu
ide.html
30/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
The Enemy is Recognized

Aliasing is shown in bottom diagrams on previous slides – Signals that are
sampled at too low a rate can reconstruct high frequencies as low frequencies

These low frequencies are “aliases” of high frequencies

The low sampling rate data could not adequately represent the high frequency
components, so it represented them incorrectly, as low frequencies

So, we just sample above the Nyquist limit, right?

Regrettably, we can’t always do that

What about this square wave signal? just a
test pattern of alternating black/white bars

Let’s try using Fourier Synthesis
Number of sine waves:
Andries van Dam©
10/1/2015
5
25
125
31/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Infinite Frequencies


Square waves have infinite frequencies at the jumps, so how can we sample
correctly at those edges?
We can’t. Pure and simple. ∞ ⋅ 2 = ∞, and we can’t sample at an infinite rate.


Unfortunately, infinite frequencies are the rule in synthetic computer graphics discrete transitions between adjacent pixels
So, do inverse operation. Instead of increasing our sampling frequency to meet
that of the signal, we:




Pre-filter out high frequencies we can’t show
The signal is now guaranteed to consist only of frequencies we can represent and
reconstruct with reasonable accuracy
This isn’t the same signal that came in, but it’s better than an aliased version
Reconstructing the pre-filtered approximate signal beats reconstructing sampled
original signal, with its corrupting aliases for the highest frequencies
Andries van Dam©
10/1/2015
32/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Infinite Frequencies

The more high frequencies we pre-filter out, the lower the sampling
frequency needed but the less the filtered signal resembles original.

Note: pre-filtering is often just abbreviated as filtering, but the prefix
“pre” helps remind us that post-filtering (i.e., another stage of filtering
after image computation or transformation) is also practiced. If postfiltering is done on reconstruction of samples of original signal, it will
blur in the aliases present in the corrupted reconstruction!
Andries van Dam©
10/1/2015
33/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Scale Aliasing, or “Why do we have to pre-filter?”
Original Image
Image with sample
points marked
Image scaled using
point samples
This doesn’t look right at all. There are no stripes and the image now has
a blacker average intensity
Andries van Dam©
10/1/2015
34/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Scale Aliasing II, or “Close, but no cigar?”
Original Image
Prefiltered image with
samples marked
Prefiltered image
scaled
Better, but not perfect…
Andries van Dam©
10/1/2015
35/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Scale Aliasing III, or “Why is it still wrong?”



Good: The pre-filtered scaled image now has same overall brightness
Bad: still no stripes
Filter removed “high frequencies” from image


discontinuities that were stripes
Given number of points to represent image once scaled, not enough
points to represent high frequencies; even higher sampling rate doesn’t
help…

Nyquist limit says that we can’t represent frequencies higher than ½ our
sampling rate, thus can’t do better than this blurred approximation
Andries van Dam©
10/1/2015
36/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Outline










Overview
Example Applications
Jaggies & Aliasing
Sampling & Duals
Convolution
Filtering
Scaling
Reconstruction
Scaling, continued
Implementation
Andries van Dam©
10/1/2015
37/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Filtering as an Application of the Convolution Integral


Remember filtering is just (weighted) averaging of samples in a neighborhood
How is it done in practice? Just a nested for loop through all the sample points





either sample at integer pixel locations or in between (e.g., for image up-scaling)
based on the theory of signal convolution – the convolution integral
we’ll use a finite approximation to that integral
First we’ll low-pass pre-filter to remove (as much as possible) un-representable
frequencies to minimize aliases (doable in most non-pathological cases for images, not for
rendering, where we’ll use “supersampling” and post-filtering – lecture IP-IV)
Second, there will be another form of filtering when the samples are “reconstructed” as a
continuous signal approximating the original signal

in the case of raster graphics, reconstruction combines


the h/w generation of pixels from pixel values as an image that feeds
the perceptual apparatus of the HVS (Human Visual System) which typically smoothes out
discontinuities (spatially and temporally), a kind of integration
Andries van Dam©
10/1/2015
38/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Low-Pass Filtering
to Eliminate High
Frequencies
(shown for one scan line in
Spatial Domain)
Andries van Dam©
10/1/2015
39/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Ideal Low-Pass Filtering (Frequency Domain)




Multiplying by the box function in the
frequency domain
Frequencies under the box are kept, and
the high frequencies are cut off
Corresponds to convolution with the sinc
function in the spatial domain
𝑠𝑖𝑛𝑐 𝑥 =
sin(𝜋𝑥)
𝜋𝑥
…
Andries van Dam©
…
10/1/2015
40/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Discrete convolution – Function as an Array


This is the type of convolution we will be using once we begin coding
If we evenly sample a function, get an array of point samples, e.g., (R,G, B)
triples as color intensity values


Some mathematicians think of an array of numbers as a function. Think this way
for the next few slides!
The larger the array (the more samples/unit interval) the more accurate the
representation of the function and the better our ability to anti-alias and
improve the reconstruction
[ 1.2, 0, -1.2, 0, 1.2, 0, -1.2 ]
Andries van Dam©
10/1/2015
41/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Discrete Convolution for Filtering




Have two arrays (functions), f and g, and we assign g as our “filter”. For the sake of
convenience, force the filter to have an odd number of points so that there is a center
Once we are done with our convolution, we will produce a new array (function)
You’ll do this for blur and edge detection, as well as for image scaling
Take the filter, and line up the center point of the filter with each element, i, of f.
Multiply each pair of elements and sum the products. Assign this value to element i
of the output array



When we are scaling, this procedure will change slightly because where we center the filter
and where our value goes will be different!
One hiccup: The output array will be slightly larger than our initial array because we
start our multiplication/sum process at the edges, where it will generate a non-zero
result
It is easier to see!
Andries van Dam©
10/1/2015
42/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Discrete Convolution – Visually (1/2)
[ 1.2, 0, -1.2, 0, 1.2, 0, -1.2 ]
Function, f
Andries van Dam©
10/1/2015
[ ⅓, ⅓, ⅓]
Function, g, a box filter
43/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Discrete Convolution – Visually (2/2)
[ 1.2, 0, -1.2, 0, 1.2, 0, -1.2 ]
[ ⅓, ⅓, ⅓ ]
[ 1.2, 0, -1.2, 0, 1.2, 0, -1.2 ]
[ ⅓, ⅓, ⅓]
[ 1.2, 0, -1.2, 0, 1.2, 0, -1.2 ]
[ ⅓, ⅓, ⅓ ]
Output
[ .4 _ _ _ _ _ _ _ _ ]
.4
[ .4 .4 _ _ _ _ _ _ _ ]
[ .4 .4 0 _ _ _ _ _ _ ]
0.0
[ 1.2, 0, -1.2, 0, 1.2, 0, -1.2 ]
[ ⅓, ⅓, ⅓ ]
Andries van Dam©
10/1/2015
[ .4 .4 0 -.4 _ _ _ _ _ ]
-.4
44/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Discrete Convolution -- Review





Think of an array as a function
We take two arrays and generate a third
We “slide” the filter along the other array and at each element, calculate a
value by multiplying the pairs and summing the products to do the
(weighted) averaging
Our output array will be slightly larger than our input array if we start too
far over the edges. We can (and will) ignore this for image processing and
just take “inner” part of the array
Another hiccup to be aware of: Our process has an additional step if our
filter is not symmetric. We will mention this more in the coming slides.
Don’t worry too much about it though, in this course most filters will be
symmetric
Andries van Dam©
10/1/2015
45/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
What is (Continuous) Convolution?

Convolution is a process that takes two function, f and g, and produces a new
function h.


For convolution, h(x) equals the integral of the product of signal f and filter g,
centered at x.


When you create a function you need to define it's value for each input, i.e. h(x) = ?
This should sound very similar to our discrete notion of convolution (integrals are like
sums)
You can think of convolution as sliding g over f to produce h because as we shift
the center of g, we evaluate h at a new point, x

in theory, an infinite number of x’s; in practice for discrete convolution, a finite number
of x’s. We’ll have to figure out where those should be in the original image, and that
will differ for up-scaling and down-scaling, as a function of scale factor
Andries van Dam©
10/1/2015
46/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Convolution Mathematically (1/2)

Convolving signal with filter function

Changing variable of integration to be 𝜏, ranging from 0 to ±∞ starting
at x
+∞
ℎ 𝑥 =𝑓 𝑥 ∗𝑔 𝑥 =
𝑓 𝜏 𝑔 𝑥 − 𝜏 𝑑𝜏
−∞
Center filter g at x
Exactly what we were
doing, just in a formula!
Multiply the pairs
Integrate (sum) the products
Andries van Dam©
10/1/2015
47/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Convolution Mathematically (2/2)


Convolving signal f(x) with filter function g(x):
At each point x, h(x) is integral of product f of and g,
except g is flipped and translated so its origin is at x



ò f (t )g(x - t )dt
-¥
In practice, f or g is often an even function (symmetric
about the y-axis) and we don’t need to flip
If g(x) has finite support, it does a weighted average
centered at x
f(x) (blue signal) convolved by g(x) (red filter) to get
result h(x) (black signal)



h(x) = f (x)* g(x) =
+¥
f(x) and g(x) are box functions
Each point on black signal is result of an integral,
represented by the area under the product of f(x) and g(x)
(yellow area)
http://www.cs.brown.edu/exploratories/fr
eeSoftware/repository/edu/brown/cs/expl
oratories/applets/convolution/convolution
_guide.html
Note: in filtered signal h(x), discontinuities of f(x) are
smoothed out and the base is broadened
Andries van Dam©
10/1/2015
48/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Simple Convolution Example
1111
×1111
1111
1111
1111
+1111
1234321
Convolution is a lot like multiplication
f (x)* f (x)
*
Try the applet:
http://www.cs.brown.edu/exploratories/freeSoftware/repository/edu/brown/cs/exploratories/
applets/twoBoxConvolution/two_box_convolution_guide.html
Andries van Dam©
10/1/2015
49/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Properties of Convolution

Commutative
𝑓∗𝑔 =𝑔∗𝑓

Associative
𝑓∗ 𝑔∗ℎ = 𝑓∗𝑔 ∗ℎ

Distributive
𝑓∗ 𝑔+ℎ = 𝑓∗𝑔 + 𝑓∗ℎ

Identity
𝑓∗𝛿 =𝑓

𝛿 is the Dirac delta– a mathematical artifice


0 everywhere except at 𝛿 0 = ∞
Area under 𝛿 is 1
𝛿 𝑥 𝑑𝑥 = 1

Can be thought of as the limit as a Gaussian narrows
Andries van Dam©
10/1/2015
50/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Duals
Sinc in spatial domain
corresponds to box/pulse in
frequency domain
Truncated sinc in spatial
domain corresponds to ringing
pulse in frequency domain –
decent approximation to
perfect pulse
Andries van Dam©
10/1/2015
51/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Common Filters and Their Duals
Box
Sinc
Spatial domain
Andries van Dam©
Bad filter in practice: gradual
attenuation, negative lobes,
infinite extent, all corrupting
signal. But still beats point
sampling!
Frequency domain
Triangle
Sinc2
Finite Gaussian
Gaussian
10/1/2015
52/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Duals
Spatial
Frequency
Multiplication
Convolution
Convolution
Multiplication
Box
Sinc
Sinc
Box
Andries van Dam©
10/1/2015
53/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Outline










Overview
Example Applications
Jaggies & Aliasing
Sampling & Duals
Convolution
Filtering
Scaling
Reconstruction
Scaling, continued
Implementation
Andries van Dam©
10/1/2015
54/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Low-Pass Filtering: Convolving with sinc


In theory sinc has infinite extent,
however small the contributions
and negative lobes, but weights
contributions at the center most
heavily.
Practically, sinc is decently
approximated with gaussian
(normal) distribution, or even
triangle, with finite extents and
weights greater than or equal to
zero.
Original signal
Sinc filter
Filter operation at origin shown as black dot
Filtered signal
Andries van Dam©
10/1/2015
55/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
What Does Filtering/Convolution Do?

Select points at which to sample, e.g., pixel centers

Slide the filter over each successive sample point and compute convolution integral at that point (the
area under product curve)

This is weighted average of current pixel and its nearby neighbors

Most useful graphics filters are symmetric about their origin and fall off rapidly from their center

Weighted average is value of pixel for filtered image

Our illustrations are 2D, but for pixels one should
think of sinc as three-dimensional weighting map


Of course, we don’t actually compute the integral, we
merely compute discrete values of filter function and
do discrete multiplication and summing to approximate.
Term “filter” used strictly in signal-processing
sense (filter can also used to mean arbitrary
transformations of an image, like in Photoshop)
Andries van Dam©
10/1/2015
Sinc in 3D
56/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Filtering Summary




Filtering is nothing more than weighted averaging around a sample point on the image
Dual of sinc function in spatial domain is box function in frequency domain
Dual of box function in spatial domain is sinc function in frequency domain
Multiplying frequency distribution with box function in frequency domain provides exact
low-pass filtering (filtering out all high frequencies past a specified one)



Hence, optimal low-pass filter is sinc in spatial domain
Conversely, convolving with a box filter in the spatial domain corresponds to multiplying with sinc
in the frequency domain, which does attenuate high frequencies but not perfectly, and introduces
artifacts because of the negative lobes
Multiplication of 𝐹(𝑥) and 𝐺(𝑥) in frequency domain corresponds to convolution of their
duals, 𝑓(𝑥) and 𝑔(𝑥), in spatial domain, and vice versa
𝐹 𝑥 𝐺 𝑥 ⇔𝑓 𝑥 ∗𝑔 𝑥
𝐹 𝑥 ∗𝐺 𝑥 ⇔𝑓 𝑥 𝑔 𝑥
Andries van Dam©
10/1/2015
57/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
Filtering Summary


To get low-pass filtering (i.e., filter out high frequencies), we often use
convolution with triangle function in spatial domain to approximate ideal
sinc
Properties of triangle function:




Easy to compute, unlike sinc, which has an infinite support (or even a Gaussian
approximation to sinc)
Its dual, sinc2, is acceptable approximation to a box function although it has
infinite extent (see slide 36)
Will cause inaccurate representation of all frequencies and therefore some degree
of corruption/aliasing must occur.
Not as bad as using box as a filter with sinc as its dual in the frequency domain

In other words, weighted area sampling of any kind, providing it is roughly coneshaped, is better than unweighted area sampling with a box filter, which is better
in turn than point sampling
Andries van Dam©
10/1/2015
58/51
CS123 | INTRODUCTION TO COMPUTER GRAPHICS
The Filtering Pipeline

To filter, we have used three-step pipeline:

Construct continuous function 𝑓 𝑥 (e.g., rectangles representing lines, polygons)


Low-pass filter function to generate 𝑓 𝑥




Geometric primitives lead to infinite frequencies at edges
Ideally, in the frequency domain: Multiply dual of 𝑓 𝑥 with box in the frequency domain
Equivalently, in spatial domain: Convolve 𝑓 𝑥 with 𝑠𝑖𝑛𝑐 (i.e., evaluate integral at an infinite number
of points)
Sample pre-filtered continuous function 𝑓 𝑥 to generate discrete function 𝑓 𝑘
As programmers, we save work:


Construct a continuous function
Simultaneously sample and filter to generate sampled image 𝑓 𝑘


Equivalent to generating new function, then evaluating it only at sample points (i.e., at pixels)
We also need to reconstruct the signal from the filtered/sampled image for display on a device

next slide deck
Andries van Dam©
10/1/2015
59/51