ESE 250:
Digital Audio Basics
Week 4
February 5, 2013
The Frequency Domain
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
1
Course Map
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
2
Musical Representation
• With this compact notation
– Could communicate a sound to pianist
– Much more compact than 44KHz time-sample
amplitudes (fewer bits to represent)
– Represent frequencies
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
3
Physics review
• Consider a sound signal that varies as a sine
wave in time: s(t )  A sin( t   )
• What is the relationship between  and f ?
–
  2f
rad/s,
f  1/ T
Hz
s (t )  A sin( 2ft   )
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
4
Time-domain Digital representation
• Week2: Quantization in TIME and VALUE
• Question: Other ways to represent this signal?
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
5
Another-domain representation?
• Consider a sound signal that varies as a sine wave in
time:
s (t )  A sin( 2ft   )
• Note that a sine wave is periodic, this implies
redundancy that we could exploit to better compress it
(week3 – compression)
• At least we could save memory by representing only
one period. But can we do even better?
• Question: What information do we need to represent
the signal?
– Amplitude, frequency, phase
• Question: How much memory to store each of these?
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
6
Another-domain representation?
• Remember week 2 lab:
• Inputs:
–
–
–
–
Amplitude: 1
Frequency: various values, 220Hz, etc…
Phase: 0
Sampling rate: 48,000 kHz
• Output: Periodic sine function sampled in time,
with 48,000 samples for 1 second of sound
• Question: Why did we need to create the large
LVM files? Why not just store the little
information we started with?
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
7
Storage advantage
• Assume each of amplitude, frequency and
phase are stored as 16bit integers.
• How much memory to store the sine in lab2?
– Only storing the info above?
– With an LVM file containing 48,000 samples?
• Percent savings?
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
8
Frequency-domain
• T = π, A = 3: s(t) = A*sin(2π*f*t) = 3*sin(2*t)
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
9
Frequency-domain
• Of course, not all signals are this simple
• For example, consider s(t) = sin(2t) + sin(t)/2
• Question: What will the frequency
representation look like?
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
10
Frequency-domain
• How about the time-domain?
– Plot sin(2t)
– Plot sin(t)/2
– Sum: sin(2t)+ sin(t)/2
• Notice how it was easier to plot
the frequency domain
representation
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
11
Frequency-domain
• Another example
• What function is this?
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
12
Frequency-domain
• So far, we have seen how a signal written as a
sum of sines of different frequencies can have
a frequency domain representation.
• Each sine component is more or less
important depending on its coefficient
• But can any arbitrary signal be represented as
a sum of sines?
– No. But the idea has potential, let’s explore it!
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
13
More background
• Fourier series:
– Any periodic signal can
be represented as a sum
of simple periodic
functions: sin and cos
sin(nt) and cos(nt)
For n = 1, 2, 3, …
These are called the
harmonics of the signal
Jean Baptiste Joseph Fourier, wikipedia
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
14
Fourier Series – more formally
Given a (well-behaved) periodic function f(t), we
can always write it as:
ao N
f (t )    [an cos( nt )  bn sin( nt )]
2 n 1
1 
Where:
N 0
an 
f (t ) cos( nt )dt



N 0
bn 
1
N 1





f (t ) sin( nt )dt
These two equations give us the a n and bn coefficients of the
Fourier series based on the input function.
These coefficients form the frequency domain representation
We will not prove the equations
If you’re interested: Office Hours, ESE 325, Math 312, …
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
15
Fourier Series – Why does it work?
• The cos(nx) and sin(nx) functions
form an orthogonal basis: they
allow us to represent any periodic
signal by taking a linear
combination of the basis
components without interfering
with one another
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
16
Orthogonal basis
• Choose a basis
• Order the vectors in it
• All vectors in the space can be represented as
sequences of coordinates, or ordered coefficients of
the basis vectors
• Example: 3D space
– Basis: [i j k]
– Linear combination: xi + yj + zk
– Coordinate representation: [x y z]
• Example: Quadratic polynomials
– Basis: [x2 x 1]
– Linear combination: ax2 + bx + c
– Coordinate representation: [a b c]
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
17
Fourier Series
• Consider the xy-plane:
10
• We can address a point by its two
coordinates (x,y)
• The blue point is located at (10, 3)
• In Fourier series, we have a basis with
infinite dimension, so we sum up over
an infinite number of harmonics
• Harmonics are 1, cos(t), cos(2t),
cos(3t), …, sin(t), sin(2t), …
5
0
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
0
5
10
18
Example of another orthogonal basis
• Although we have an infinite number of basis
elements, we don’t need to use all of them
• The components with the largest coefficients are the
most significant
• The more components we add, the closer to the
function we get.
• i.e.: As
N   , error  0
Examples follow
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
19
Fourier Series – Sawtooth wave
(falstad.com/fourier/)
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
20
Fourier Series – Sawtooth wave
(falstad.com/fourier/)
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
21
Interlude
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
22
Where Are We Heading After Today?
• Week 2
– Received signal is
• discrete-time-stamped
• quantized
– q = PCM[ r ]
= quantL [SampleTs[r] ]
Generic Digital Signal Processor
Sample
r(t)
q
Code
c
Store/
Transmit
Decode
Produce
p(t)
• Week 3
– Quantized Signal is Coded
– c =code[ q ]
• Week 4
– Sampled signal
• not coded directly
• First linearly
transformed into
frequency domain
– Q = DFT[ q ]
Psychoacoustic Audio Coder
q
Q
c
[Painter & Spanias. Proc.IEEE, 88(4):451–512, 2000]
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
23
What now?
• In the first half of the lecture we have
introduced the idea of frequency domain
• In the second half of the lecture, we will see
how to extend the idea of Fourier Series to the
class of signals we are interested in:
• Not necessarily periodic and quantized
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
24
Frequency-domain
•
•
•
•
Fourier Series deal with periodic signals
Are all signals periodic?
Is your favorite mp3 song periodic?
What do we do with these?
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
25
Frequency-domain
• Certainly, we could loop a song and make the
signal periodic, but this is not a good solution
• Mathematicians have extended the notion of
Fourier Series into that of Fourier Transforms
by making the period of the signal go to
infinity: T  
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
26
Discrete Fourier Transforms
• Fourier Transforms are nice, but we want to store and
process our signals with computers
• We therefore extend Fourier Transforms into
Discrete Fourier Transforms, or DFT
• The signal is now discrete: x(t )  xn
• The signal contains N samples: 0  n  N 1
it
e
 cos(t )  i sin( t )
• Remember Euler’s formula:
N 1
DFT ( xn )  X k   xn  e
The frequency domain
i 2kn / N signal X k also has
N-1 elements: 0  k  N 1
n 0
1
DFT ( X k )  xn 
N
1
N 1
X
k 0
k
e
The time domain
x
i 2kn / N signal n can be
reconstructed
from X k
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
27
Discrete Fourier Transforms
• A DFT transforms N samples of a signal in time domain into
a (periodic) frequency representation with N samples
• So we don’t have to deal with real signals anymore
• We work with sampled signals (quantized in time), and the
frequency representation we get is also quantized in time!
N 1
DFT ( xn )  X k   xn  e i 2kn / N
n 0
(sepwww.stanford.edu/oldsep/hale/FftLab.html)
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
28
Discrete Fourier Transforms
• A smaller sampling period
-> More points to represent the signal
larger N
-> More harmonics used in DFT
N harmonics
-> Smaller error compared to actual analog
signal we capture/produce
• DFTs are extensively used in practice, since
computers can handle them
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
29
Discrete Fourier Transform exercise
N 1
DFT ( xn )  X k   xn  e i 2kn / N
n 0
N 1
2k
2k
X k   xn  cos( n
)  i  xn  sin( n
)
N
N
n 0
n 0
N 1
• Complete the table below using the DFT equation shown above
(The bottom
is redundant
because the
DFT of a real
signal is
symmetrical)
7 samples -> 7
harmonics
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
30
Approximating the Sampled Signal
• Approximate Reconstruction
– Always achievable
• A signal sampled in time can be
approximated arbitrarily closely from
the time-sampled values
• With a DFT, each sample gives us
knowledge of one harmonic
• Each harmonic is a component used in
the reconstruction of the signal
• The more harmonics we use, the better
the reconstruction
{ Cos[0t], Sin[1t], Cos[1t], Sin[2t], Cos[2t], Sin[3t], Cos[3t] }
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
31
Usually Computed, Not “Solved”
7 Samples; 7 Harmonics
DFT
11 Samples ; 11 Harmonics
DFT
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
15 Samples; 15 Harmonics
DFT
32
Measured Data:
Yet Another Sampled
(Real) Signal
Sampled Signal:
v
t
ESE 250 – Spring’10 DeHon & Koditschek
33
Some Signals Dislike Some
Harmonics
• Approximate Reconstruction
15 Samples & Harmonics
– although always achievable
– may require a lot of samples
– to get good performance
Sometimes time is better than
frequency
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
21 Samples & Harmonics
31 Samples & Harmonics34
Saving resources
•
•
•
•
However, N can get very large
e.g., with a sampling rate of 48,000Hz
How big is N for a 4min song?
How many operations does this translate to?
– To compute one frequency component?
– To compute all N frequency components?
• This is not practical. Instead, we use a window
of values to which we apply the transform
– Typical size: …, 512, 1024, 2048, …
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
35
A Window Operation
• In the example below, we traverse the signal
but only look at 64 samples at a time
Time
Freq
(sepwww.stanford.edu/oldsep/hale/FftLab.html)
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
36
A Window Operation
• The following shows a DFT (on the right) applied to a sinusoidal
signal (on the left). The DFT is only applied to part of the signal, the
window, which is shown in the middle
• When we increase the window size, the information of the DFT is
more accurate
16-point
32-point
8-point DFT
DFT
(http://math.mit.edu/mathlets/mathlets/discrete-fourier-transform/)
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
37
Moonlight Sonata in Time and Frequency
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
38
Big ideas
• A signal in time-domain can also be represented in the frequency
domain.
– X(f) = Transform(x(t))
• This representation may be more efficient:
– For storage
– For signal processing
– For understanding the wave (e.g. can cut-off some frequencies)
• We go from a signal represented by values in time to one
represented by coefficients of harmonics
• Fourier Series, for continuous periodic signals
• FT: Fourier Transform, for continuous signals
• DFT: Discrete Fourier Transform, for discrete signals
– DFTs are extensively used in practice
– Finite amount of data, easy to implement for computers
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
39
Admin
• Lab 3 due Wednesday (tomorrow)
• On Thursday, Lab 4:
– You will be looking at signals in the frequency
domain in LabView
– You will create your own synthesizer!
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
40
Announcement
• Hackathon and bootcamp
– If you are interested
• The bootcamp will be this Saturday, noon to
six, in Detkin lab.
• The Hackathon will begin around 5-6 the
weekend of the 15-17th
ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah
41