ALARI/DSP
INTRODUCTION-2
Toon van Waterschoot & Marc Moonen
Dept. E.E./ESAT, K.U.Leuven
toon.vanwaterschoot@esat.kuleuven.be
http://homes.esat.kuleuven.be/~tvanwate
DSP-II
p. 1
INTRODUCTION-1 : Overview
• Introduction
• Discrete-time signals
sampling, quantization, reconstruction
• Stochastic signal theory
deterministic & random signals, (auto-)correlation functions,
power spectra, …
• Discrete-time systems
LTI, impulse response, FIR/IIR, causality & stability,
convolution & filtering, …
• Complex number theory
complex numbers, complex plane, complex sinusoids,
circular motion, sinusoidal motion, …
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 2
INTRODUCTION-2 : Overview
• z-transform and Fourier transform
region of convergence, causality & stability, properties,
frequency spectrum, transfer function, pole-zero
representation, …
• Elementary digital filters
shelving filters, presence filters, all-pass filters
• Discrete transforms
DFT, FFT, properties, fast convolution,
overlap-add/overlap-save, …
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 3
z- and Fourier-transform: overview
• z-transform:
– definition & properties
– complex variables
– region of convergence
• Fourier transform:
– frequency response
– Fourier transform
• Transfer functions:
–
–
–
–
ALARI/DSP
difference equations
rational transfer functions
poles & zeros
stability in the z-domain
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 4
z- and Fourier-transform: z-transform
• definition:
discrete-time sequence in integer variable
z-transform
discrete-time series in complex variable
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 5
z- and Fourier-transform: z-transform
• definition:
– z-transform of a discrete-time signal:
z-transform
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 6
z- and Fourier-transform: z-transform
• definition:
– z-transform of a discrete-time system impulse response:
z-transform
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 7
z- and Fourier-transform: z-transform
• properties:
– linearity property:
– time-shift theorem:
– convolution theorem:
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 8
z- and Fourier-transform: z-transform
• region of convergence:
– the z-transform of an infinitely long sequence is a series
with an infinite number of terms
– for some values of the series may not converge
– the z-transform is only defined within the region of
convergence (ROC):
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 9
z- and Fourier-transform: Fourier transf.
• Frequency response:
– for an LTI system a sinusoidal input signal
produces a sinusoidal output signal at the same frequency
– the output can be calculated from the convolution:
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 10
z- and Fourier-transform: Fourier transf.
• Frequency response:
– the sinusoidal I/O relation is
– the system’s frequency response
complex function of the radial frequency
ALARI/DSP
•
denotes the magnitude response
•
denotes the phase response
May 2013
Toon van Waterschoot & Marc Moonen
is a
:
INTRODUCTION-2
p. 11
z- and Fourier-transform: Fourier transf.
• Frequency response:
– the frequency response
is equal to the ztransform of the system’s impulse response, evaluated
at
– for
,
is a complex function describing
the unit circle in the z-plane
Im
z-plane
Re
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 12
z- and Fourier-transform: Fourier transf.
• Frequency response & Fourier transform
j
– the frequency response H (e ) of an LTI system is equal to the
Fourier transform of the continuous-time impulse sequence
constructed with h[k] :

F{hD (t )}  F{  h[k ]. (t  k.Ts )}  ...  H (e j ) ,   2 .
k  
f
fs
– similarly, the frequency spectrum of a discrete-time signal U (e j ), Y (e j )
(=its z-transform evaluated at the unit circle) is equal to the Fourier
transform of the continuous-time impulse sequence constructed
with u[k], y[k] :

f
F{uD (t )}  F{  u[k ]. (t  k.Ts )}  ...  U (e j ) ,   2 .
fs
k  
• Input/output relation:
ALARI/DSP
May 2013
Y (e j )  H (e j ).U (e j )
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 13
z- and Fourier-transform: Transfer func.
• Difference equations:
– the I/O behaviour of an LTI system using an FIR model,
can be described by a difference equation:
– the I/O behaviour of an LTI system using an IIR model,
can be described by a difference equation with an
autoregressive part in the left-hand side:
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 14
z- and Fourier-transform: Transfer func.
• Rational transfer functions:
– transforming the FIR difference equation to the zdomain and using the convolution theorem, leads to:
– the z-transform
of the impulse response is called
the transfer function of the system:
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 15
z- and Fourier-transform: Transfer func.
• Rational transfer functions:
– transforming the IIR difference equation to the z-domain
and using the convolution theorem, leads to:
– the ratio of
and
is equal to the z-transform
of the impulse response and is called the transfer
function of the system:
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 16
z- and Fourier-transform: Transfer func.
• Poles and zeros:
– the zeros of a rational transfer function are defined as
the roots of the nominator polynomial
– the poles of a rational transfer function are defined as
the roots of the denominator polynomial
Im
z-plane
– e.g.
Re
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 17
z- and Fourier-transform: Transfer func.
• Stability in the z-domain:
– the pole-zero representation of a rational transfer
function allows for an easy stability check
– an LTI system is stable if all of its poles lie inside the
unit circle in the complex z-plane
Im
ALARI/DSP
May 2013
Im
stable
unstable
Re
Re
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 18
INTRODUCTION-2 : Overview
• z-transform and Fourier transform
region of convergence, causality & stability, properties,
frequency spectrum, transfer function, pole-zero
representation, …
• Elementary digital filters
shelving filters, presence filters, all-pass filters
• Discrete transforms
DFT, FFT, properties, fast convolution,
overlap-add/overlap-save, …
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 19
Elementary digital filters: overview
• Shelving filters:
– definition
– one-zero
– one-pole
• Presence filters:
–
–
–
–
definition
two-zero
two-pole
biquadratic
• All-pass filters:
– definition
– biquadratic
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 20
Elementary digital filters: shelving filters
• Definition:
– a shelving filter is a filter that amplifies a signal in the
frequency range
Hz (boost), while attenuating it
in the range
Hz (cut), or vice versa
• Low-pass filter:
– low-frequency boost, high-frequency cut
• High-pass filter:
– low-frequency cut, high-frequency boost
• Cut-off frequency:
– the cut-off frequency
is usually defined as the
frequency at which the filter gain is 3dB less than the
gain at Hz (low-pass) or
Hz (high-pass)
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 21
Elementary digital filters: shelving filters
• One-zero shelving filter:
– difference equation:
– transfer function:
– signal flow graph:

ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 22
Elementary digital filters: shelving filters
• One-zero shelving filter:
– 1 real zero:
– highpass if
– lowpass if
Im
ALARI/DSP
May 2013
Im
highpass
lowpass
Re
Re
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 23
Elementary digital filters: shelving filters
• One-zero shelving filter:
– frequency response
– frequency magnitude response:
– frequency phase response:
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 24
Elementary digital filters: shelving filters
• One-zero shelving filter:
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 25
Elementary digital filters: shelving filters
• One-pole shelving filter:
– difference equation:
– transfer function:
– signal flow graph:

ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 26
Elementary digital filters: shelving filters
• One-pole shelving filter:
– 1 real pole:
– highpass if
– lowpass if
Im
ALARI/DSP
May 2013
Im
highpass
lowpass
Re
Re
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 27
Elementary digital filters: shelving filters
• One-pole shelving filter:
– frequency response
– frequency magnitude response:
– frequency phase response:
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 28
Elementary digital filters: shelving filters
• One-pole shelving filter:
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 29
Elementary digital filters: presence filters
• Definition:
– a presence filter is a filter that amplifies a signal in the
frequency range around a center frequency
Hz
(boost), while attenuating elsewhere (cut), or vice versa
• Resonance filter:
– boost at center frequency (band-pass)
• Notch filter:
– cut at center frequency (band-stop)
• Bandwidth:
– the bandwidth
is defined as the frequency difference
between the frequencies at which the filter gain is 3dB
lower/higher than the resonance/notch gain
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 30
Elementary digital filters: presence filters
• Two-zero presence filter:
– diff. eq.:
– transfer function:
– signal flow graph:


ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 31
Elementary digital filters: presence filters
• Two-zero presence filter:
– 2 zeros:
– if
: real zeros  cascade shelving filters
– if
: complex conj. zero pair  notch filter
cascade
shelving
filters
Im
Im
notch filter
Re
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
Re
INTRODUCTION-2
p. 32
Elementary digital filters: presence filters
• Two-zero notch filter:
– transfer function in radial representation:
– radial center frequency
Im
notch filter
– zero radius
Re
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 33
Elementary digital filters: presence filters
• Two-zero notch filter:
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 34
Elementary digital filters: presence filters
• Two-pole presence filter:
– diff. eq.:
– transfer function:
– signal flow graph:


ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 35
Elementary digital filters: presence filters
• Two-pole presence filter:
– 2 poles:
– if
: real poles  cascade shelving filters
– if
: comp. conj. pole pair  resonance filter
cascade
shelving
filters
Im
Im
resonance filter
Re
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
Re
INTRODUCTION-2
p. 36
Elementary digital filters: presence filters
• Two-pole resonance filter:
– transfer function in radial representation:
– radial center frequency
– pole radius
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
Im
resonance filter
Re
INTRODUCTION-2
p. 37
Elementary digital filters: presence filters
• Two-pole resonance filter:
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 38
Elementary digital filters: presence filters
• Biquadratic presence filter:
– difference equation:
– transfer function:
– 2 poles:
– 2 zeros:
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 39
Elementary digital filters: presence filters
• Biquadratic presence filter:
– signal flow graph:


ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 40
Elementary digital filters: presence filters
• Constrained biquadratic presence filter:
constrained
biquadratic
resonance
filter
Im
Im
Re
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
constrained
biquadratic
notch filter
Re
INTRODUCTION-2
p. 41
Elementary digital filters: all-pass filters
• Definition:
– a (unity-gain) all-pass filter is a filter that passes all input
signal frequencies without gain or attenuation
– hence a (unity-gain) all-pass filter preserves signal energy
– an all-pass filter may have any phase response
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 42
Elementary digital filters: all-pass filters
• Biquadratic all-pass filter:
– it can be shown that for the unity-gain constraint to hold,
the denominator coefficients must equal the numerator
coefficients in reverse order, e.g.,
– the poles and zeros are moreover related as follows
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 43
INTRODUCTION-2 : Overview
• z-transform and Fourier transform
region of convergence, causality & stability, properties,
frequency spectrum, transfer function, pole-zero
representation, …
• Elementary digital filters
shelving filters, presence filters, all-pass filters
• Discrete transforms
DFT, FFT, properties, fast convolution,
overlap-add/overlap-save, …
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 44
Discrete transforms: overview
• Discrete Fourier Transform (DFT):
–
–
–
–
definition
inverse DFT
matrix form
properties
• Fast Fourier Transform (FFT):
• Digital filtering using the DFT/FFT:
–
–
–
–
ALARI/DSP
linear & circular convolution
overlap-add method
overlap-save method
fast convolution
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 45
Discrete transforms: DFT
• DFT definition:
– the Fourier transform of a signal or system is a
continuous function of the radial frequency :
– the Fourier transform can be discretized by sampling it
at
discrete frequencies
between
and
, uniformly spaced
:
= DFT
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 46
Discrete transforms: DFT
• Inverse discrete Fourier transform (IDFT):
– an -point DFT can be calculated from an
sequence:
-point time
– vice versa, an -point time sequence can be calculated
from an -point DFT:
= IDFT
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 47
Discrete transforms: DFT
• matrix form
– using the shorthand notations
the DFT and IDFT definitions can be rewritten as:
DFT:
IDFT:
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 48
Discrete transforms: DFT
• matrix form
– the DFT coefficients
be calculated as
– an
ALARI/DSP
-point DFT requires
May 2013
Toon van Waterschoot & Marc Moonen
can then
complex multiplications
INTRODUCTION-2
p. 49
Discrete transforms: DFT
• matrix form
– the IDFT coefficients
be calculated as
– an
ALARI/DSP
-point IDFT requires
May 2013
Toon van Waterschoot & Marc Moonen
can then
complex multiplications
INTRODUCTION-2
p. 50
Discrete transforms: DFT
• properties:
– linearity & time-shift theorem (cf. z-transform)
– frequency-shift theorem (modulation theorem):
– circular convolution theorem:
if
and
are periodic with period
, then
(see also ‘Digital filtering using the DFT/FFT’)
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 51
Discrete transforms: FFT
• Fast Fourier Transform (FFT)
split up N-point DFT in two N/2-point DFT’s
split up two N/2-point DFT’s in four N/4-point DFT’s
…
split up N/2 2-point DFT’s in N 1-point DFT’s
calculate N 1-point DFT’s
rebuild N/2 2-point DFT’s from N 1-point DFT’s
…
rebuild two N/2-point DFT’s from four N/4-point DFT’s
rebuild N-point DFT from two N/2-point DFT’s
– DFT complexity of
multiplications is reduced to
FFT complexity of
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
John W.Tukey
Carl Friedrich Gauss (1777-1855
•
•
•
•
•
•
•
•
•
James W. Cooley
– divide-and-conquer approach:
multiplications
INTRODUCTION-2
p. 52
Discrete transforms: Digital filtering
• Linear and circular convolution:
– circular convolution theorem: due to the sampling of the
frequency axis, the IDFT of the product of two -point
DFT’s corresponds to the circular convolution of two
length- periodic signals
– LTI system: the output sequence is the linear
convolution of the impulse response with the input signal
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 53
Discrete transforms: Digital filtering
• Linear and circular convolution:
– the linear convolution of a lengthimpulse
response
with a length- input signal
is equivalent to their -point circular convolution
if both sequences are zero-padded to length :
zero padding
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 54
Discrete transforms: Digital filtering
• Overlap-add/overlap-save:
– in many applications the input sequence length
is
much larger than the impulse response length
– computing the DFT of a very long or even infinitely long
sequence is not feasible
– a block-based convolution method is more
appropriate:
• the input sequence is divided in relatively short blocks
• each input block is circularly convolved with the impulse
response using the DFT approach
• the output signal is reconstructed using the overlap-add
method or the overlap-save method
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 55
Discrete transforms: Digital filtering
• Overlap-add method:
…
+
+
+…
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 56
Discrete transforms: Digital filtering
• Overlap-save method:
=
=
…
discard
discard
…
discard
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 57
Discrete transforms: Digital filtering
• Fast convolution:
– if in the above digital filtering methods the DFT is
implemented using an FFT algorithm, then so-called
fast convolution methods are obtained
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 58
Hungry for more?
• Some nice introductory books:
– S. J. Orfanidis, “Introduction to Signal Processing”, Prentice-Hall
Signal Processing Series, 798 p., 1996
– J. H. McClellan, R. W. Schafer, and M. A. Yoder, “DSP First: A
Multimedia Approach”, Prentice-Hall, 1998
– P. S. R. Diniz, E. A. B. da Silva and S. L. Netto, “Digital Signal
Processing: System Analysis and Design”, Cambridge University
Press, 612 p., 2002
• Some interesting online books:
– Smith, J.O. Mathematics of the Discrete Fourier Transform (DFT),
http://ccrma.stanford.edu/~jos/mdft/, 2003, ISBN 0-9745607-0-7.
– Smith, J.O. Introduction to Digital Filters, August 2006 Edition,
http://ccrma.stanford.edu/~jos/filters06/.
ALARI/DSP
May 2013
Toon van Waterschoot & Marc Moonen
INTRODUCTION-2
p. 59