Lecture 2
Signals & Systems Review
Marc Moonen & Toon van Waterschoot
Dept. E.E./ESAT, K.U.Leuven
marc.moonen@esat.kuleuven.be
www.esat.kuleuven.be/scd/
Signals & Systems Review
• Discrete-Time/Digital Signals
sampling, quantization, reconstruction
• Discrete-Time Systems
LTI, impulse response, convolution, z-transform, frequency
response, frequency spectrum, IIR/FIR
• Discrete/Fast Fourier Transform
(I)DFT, FFT
DSP-II
Version 2009-2010
Lecture-2: Signals & Systems Review
p. 2
here…
Introduction: Digital Signal Processing?
• Signal = a physical quantity that varies as a function of
some independent variable(s), e.g. time, position,
frequency, …
– 1-dimensional: speech signal, audio signal,
electromagnetic/radio signal, …
– 2-dimensional: image, …
– N-dimensional
• Processing = `filtering’ (mostly) = noise reduction,
equalization, signal separation, …
• Digital = …in `digital domain’
DSP-II
Version 2009-2010
Lecture-2: Signals & Systems Review
p. 3
Introduction: Digital Signal Processing?
Joseph Fourier (1768-1830)
Analog signal processing
Analog Domain
(Continuous-Time Domain)
Analog
Signal
Processing
Circuit
Analog IN
Analog OUT
y (t )
u (t )


U ( f )   u (t ).e  j 2 . f .t dt

Y( f ) 
 j 2 . f .t
y
(
t
).
e
dt


(=Spectrum/Fourier Transform)
DSP-II
Version 2009-2010
Lecture-2: Signals & Systems Review
p. 4
Introduction: Digital Signal Processing?
Digital signal processing in an analog world
Analog
domain
Analog IN
DSP-II
Digital
domain
Analog-toDigital
Conversion
011010
0101
Analog
domain
100110
0010
DSP
Digital
IN
Version 2009-2010
Digital
OUT
Digital-toAnalog
Conversion
Analog OUT
Lecture-2: Signals & Systems Review
p. 5
Discrete-Time/Digital Signals
Analog IN
Analog-toDigital
Conversion
011010
0101
100110
0010
DSP
Digital
IN
sampling
& quantization
DSP-II
Version 2009-2010
Digital
OUT
1/10
Digital-toAnalog
Conversion
Analog OUT
reconstruction
Lecture-2: Signals & Systems Review
p. 6
Discrete-Time/Digital Signal 2/10 : Sampling
• Time-domain sampling
x(t )
amplitude
amplitude
continuous-time
signal
x[k ]  x(k .Ts )
discrete-time
signal
impulse
train
Ts
continuous-time (t)
01234
discrete-time [k]
It will turn out (page 27) that a spectrum can be
computed from x[k], which (remarkably) will be

equal to the spectrum (Fourier transform) of the xD (t )  x(t ).
 (t  k .Ts )
(continuous-time) sequence of impulses =
k  

DSP-II
Version 2009-2010
Lecture-2: Signals & Systems Review
p. 7
Discrete-Time/Digital Signals 3/10 : Sampling
• Spectrum replication
– time domain:

xD (t )  x(t ).   (t  k .Ts )
k  
– frequency domain:
1 
k
XD( f )  .  X ( f  )
Ts k 
Ts
X(f )
XD( f )
magnitude
frequency (f)
DSP-II
Version 2009-2010
magnitude
frequency (f)
Lecture-2: Signals & Systems Review
p. 8
Discrete-Time/Digital Signals 4/10 : Sampling
• Sampling theorem
– analog signal spectrum X(f) has a bandwidth of fmax Hz
– spectrum replicas are separated by fs =1/Ts Hz
magnitude
frequency
– no spectral overlap if and only if
DSP-II
Version 2009-2010
Lecture-2: Signals & Systems Review
p. 9
Harry Nyquist (7 februari 1889 – 4 april 1976)
Discrete-Time/Digital Signals 5/10 : Sampling
• Sampling theorem
– terminology:
• sampling frequency/rate fs
• Nyquist frequency fs/2
• sampling interval/period Ts
– e.g. CD audio: fs = 44,1 kHz
• Anti-aliasing prefilters
– if
then frequencies above the Nyquist
frequency are ‘folded’ into lower frequencies (=aliasing)
– to avoid aliasing, sampling is usually preceded by a lowpass (=anti-aliasing) filtering
DSP-II
Version 2009-2010
Lecture-2: Signals & Systems Review
p. 10
Discrete-Time/Digital Signals 6/10 : Quantization
• B-bit quantization
amplitude
discrete-time signal
x[k ]
quantized discrete-time signal
amplitude
=digital signal
xQ [k ]
3Q
2Q
Q
0
-Q
-2Q
-3Q
R
discrete time [k]
discrete time [k]
number of bits B  log 2 (
DSP-II
Version 2009-2010
range R
 1)
quantizati on width Q
Lecture-2: Signals & Systems Review
p. 11
Discrete-Time/Digital Signals 7/10 : Quantization
• B-bit quantization:
– the quantization error
take on values between
can only
and
– hence
can be considered as a random noise
signal (see below) with range
– the signal-to-noise ratio (SNR) of the B-bit quantizer can
then be defined as the ratio of the signal range and
the quantization noise range
:
= the “6dB per bit” rule
DSP-II
Version 2009-2010
Lecture-2: Signals & Systems Review
p. 12
Discrete-Time/Digital Signals 8/10 : Reconstruction
• Reconstructor =
– ‘fill the gaps’ between adjacent samples
– e.g. staircase reconstructor (with `hold’ circuit):
amplitude
x[k ]
discrete-time/digital
signal
discrete time [k]
DSP-II
Version 2009-2010
amplitude xR (t )
reconstructed
analog signal
continuous time (t)
Lecture-2: Signals & Systems Review
p. 13
Discrete-Time/Digital Signals 9/10 : Reconstruction
• Ideal reconstructor =
– Ideal (rectangular) low-pass filter
– no distortion
XD( f )
magnitude
frequency (f)
• Staircase reconstructor =
– sinc-like low-pass filter
with sidelobes
– distortion due to
spurious high
frequencies
DSP-II
Version 2009-2010
XD( f )
magnitude
frequency
frequency(f)
Lecture-2: Signals & Systems Review
p. 14
Discrete-Time/Digital Signals 10/10 : Reconstruction
• Anti-image post-filter
– low-pass filter succeeds reconstructor, to remove
spurious high frequency components due to non-ideal
reconstruction
• Complete scheme is…
Analog IN
Analog OUT
antialiasing
prefilter
antiimage
postfilter
DSP-II
sampler
quantizer
DSP
Digital
IN
Version 2009-2010
reconstructor
Digital
OUT
Lecture-2: Signals & Systems Review
p. 15
Discrete-Time Systems 1/15
Discrete-time (DT) system is `sampled data’ system
u[k]
y[k]
Input signal u[k] is a sequence of samples (=numbers)
..,u[-2],u[-1] ,u[0], u[1],u[2],…
System then produces a sequence of output samples y[k]
..,y[-2],y[-1] ,y[0], y[1],y[2],…
Example: `DSP’ block in previous slide
DSP-II
Version 2009-2010
Lecture-2: Signals & Systems Review
p. 16
Discrete-Time Systems 2/15
Will consider linear time-invariant (LTI) DT systems
u[k]
y[k]
Linear :
input u1[k] -> output y1[k]
input u2[k] -> output y2[k]
hence a.u1[k]+b.u2[k]-> a.y1[k]+b.y2[k]
Time-invariant (shift-invariant)
input u[k] -> output y[k],
DSP-II
hence input u[k-T] -> output y[k-T]
Version 2009-2010
Lecture-2: Signals & Systems Review
p. 17
Discrete-Time Systems 3/15
Causal systems
iff for all input signals with u[k]=0,k<0 -> output y[k]=0,k<0
Impulse response
input …,0,0, 1 ,0,0,0,...-> output …,0,0, h[0] ,h[1],h[2],h[3],...
General input u[0],u[1],u[2],u[3]
(cfr. linearity & shift-invariance!)
0
0
0 
 y[0]  h[0]
 y[1]   h[1] h[0]
 u[0]
0
0


 

 y[2] h[2] h[1] h[0]
0   u[1] 


.
h[2] h[1] h[0] u[2]
 y[3]  0


 y[4]  0
0
h[2] h[1]  u[3]

 

0
0
h[2]
 y[5]  0
DSP-II
Version 2009-2010
this is called a
`Toeplitz’ matrix
Lecture-2: Signals & Systems Review
p. 18
Discrete-Time Systems 4/15
Convolution
u[0],u[1],u[2],u[3]
0
0 
 y[0] h[0] 0
 y[1]   h[1] h[0] 0
 u[0]
0


 

 y[2] h[2] h[1] h[0] 0   u[1] 



.
 y[3]  0 h[2] h[1] h[0] u[2]


 y[4]  0
0 h[2] h[1]  u[3]

 

0
0 h[2]
 y[5]  0
y[0],y[1],...
h[0],h[1],h[2],0,0,...
y[k ]   h[k  i ].u[i ]  h[k ] * u[k ] = `convolution sum’
i
DSP-II
Version 2009-2010
Lecture-2: Signals & Systems Review
p. 19
Discrete-Time Systems 5/15
Z-Transform of system h[k] and signals u[k],y[k]

H ( z )   h[i ]. z
i

U ( z )   u[i ]. z
i
i

Y ( z )   y[i ]. z i
i
i
0
0
0 
 y[0]
 h[0]
 y[1] 
 h[1] h[0]
0
0  u[0]



 y[2]
h[2] h[1] h[0]
0   u[1] 
1
2
3
4
5
1
2
3
4
5
1 z
z
z
z
z .
z
z
z
z .
1 z
.
y
[
3
]
0
h
[
2
]
h
[
1
]
h
[
0
]



 u[2]


 y[4]
 0
0
h[2] h[1]  u[3]




0
0
h[2]
 y[5]
 0
 




H ( z).1 z1





Y ( z)
 Y ( z )  H ( z ).U ( z )
DSP-II
Version 2009-2010
z2
z3





H(z) is `transfer function’
Lecture-2: Signals & Systems Review
p. 20
Discrete-Time Systems 6/15
Z-Transform
• easy input-output relation: Y ( z )  H ( z ).U ( z )
• may be viewed as `shorthand’ notation
(for convolution operation/Toeplitz-vector product)
• stability
=bounded input u[k] leads to bounded output y[k]
--iff
 h[k ]  
k
--iff poles of H(z) inside the unit circle
(now z=complex variable)
(for causal,rational systems, see below)
DSP-II
Version 2009-2010
Lecture-2: Signals & Systems Review
p. 21
Discrete-Time Systems 7/15
Example-1 : `Delay operator’
u[k]

Impulse response is …,0,0 ,0, 1,0,0,0,…
Transfer function is
Pole at z=0
H ( z)  z
1
u[k]
Example-2 : Delay + feedback
Impulse response is …,0,0 ,0, 1,a,a^2,a^3…
Transfer function is
Pole at z=a
H ( z )  z 1  a.z 2  a 2 .z 3  a 3 .z 4  ...
1
y[k]=u[k-1]
 H ( z )  a.z H ( z )  z
1
+
y[k]

x
a
z 1
1
 H ( z) 

1  a.z 1 z  a
DSP-II
Version 2009-2010
Lecture-2: Signals & Systems Review
p. 22
Discrete-Time Systems 8/15
Will consider only rational transfer functions:
B( z ) b0 z N  b1 z N 1  ...  bN
b0  b1 z 1  ...  bN z  N
H ( z) 
 N

N 1
A( z )
z  a1 z
 ...  a N z
1  a1 z 1  ...  a N z  N
• In general, these represent `infinitely long impulse response’ (`IIR’)
systems
• N poles (zeros of A(z)) , N zeros (zeros of B(z))
• corresponds to difference equation
Y ( z )  H ( z ).U ( z )  A( z ).Y ( z )  B( z ).U ( z )  ...
y[k ]  b0 .u[k ]  b1.u[k  1]  ...  bN .u[k  N ]  a1. y[k  1]  ...  aN . y[k  N ]
• Hence rational H(z) can be realized with finite number of delay
elements, multipliers and adders
DSP-II
Version 2009-2010
Lecture-2: Signals & Systems Review
p. 23
Discrete-Time Systems 9/15
Special case is
B( z )
1
N
H ( z) 

b

b
z

...

b
z
0
1
N
zN
• N poles at the origin z=0 (hence guaranteed stability)
• N zeros (zeros of B(z))
= `all zero’ filters
• corresponds to difference equation
Y ( z )  H ( z ).U ( z )  y[k ]  b0 .u[k ]  b1.u[k  1]  ...  bN .u[k  N ]
=`moving average’ (MA) filters
• impulse response is
0,0,0, b0 , b1,..., bN 1, bN ,0,0,0,...
= `finite impulse response’ (`FIR’) filters
DSP-II
Version 2009-2010
Lecture-2: Signals & Systems Review
p. 24
Discrete-Time Systems 10/15
Im
H(z) & frequency response:
u[2]
• given a system H(z)
• given an input signal = complex sinusoid
u[k ]  e jk

Re
Im
  k  
u[1]
u[0]=1
 cos(k )  j. sin( k )
• output signal :
Re
y[k ]   h[i ].u[ k i ]   h[i ].e j ( k i ) e jk  h[i ].e ji  u[k ].H (e j )
i
i
j
H (e )
DSP-II
i
= `frequency response’
= complex function of radial frequency ω
= H(z) evaluated on the unit circle
Version 2009-2010
Lecture-2: Signals & Systems Review
p. 25
Discrete-Time Systems 11/15
H(z) & frequency response:
• periodic : period =
2
• for a real impulse response h[k]
Magnitude response H (e j )
Phase response
j
H (e )
j
H (e )
is even function
is odd function
• example (`low pass filter’):
Nyquist frequency
1
e jk  ...,1,1,1,1,1,...
0.5
0
-4
5

-2
0
2

4

(=2 samples/period)
0
DSP-II
-5
-4
-2
0
2
Version
2009-2010
4
Lecture-2: Signals & Systems Review
p. 26
Discrete-Time Systems 12/15
• H(z) & Fourier transform
– the frequency response H (e j ) of an LTI system is equal to the
Fourier transform of the continuous-time impulse sequence (see
p.7) constructed with h[k] :

F{hD (t )}  F{  h[k ]. (t  k.Ts )}  ...  H (e j ) ,   2 .
k  
f
fs
– similarly, the frequency spectrum of a discrete-time signal U (e j ), Y (e j )
(=its z-transform evaluated at the unit circle) is equal to the Fourier
transform of the continuous-time impulse sequence constructed
with u[k], y[k] :

f
F{u D (t )}  F{  u[k ]. (t  k.Ts )}  ...  U (e j ) ,   2 .
fs
k  
• Input/output relation:
DSP-II
Version 2009-2010
Y (e j )  H (e j ).U (e j )
Lecture-2: Signals & Systems Review
p. 27
Discrete-Time Systems 13/15
• Example: All-pass filter
– a (unity-gain) all-pass filter is a filter that passes all input
signal frequencies without gain or attenuation
H (e
j
) 1
– hence a (unity-gain) all-pass filter preserves signal energy

 u[k ]
2

k  

 y[k ]
2
k  
– an all-pass filter may have any phase response
DSP-II
Version 2009-2010
Lecture-2: Signals & Systems Review
p. 28
Discrete-Time Systems 14/15
• Example: Biquadratic (=2nd order) all-pass filter
– it can be shown that for the unity-gain constraint to hold,
the denominator coefficients must equal the numerator
coefficients in reverse order, i.e.,
– the poles and zeros are then related as follows
DSP-II
Version 2009-2010
Lecture-2: Signals & Systems Review
p. 29
Discrete-Time Systems 15/15
• PS:
– So far have only considered single-input/single-output
(SISO) systems
Y ( z )  H ( z ).U ( z )
– Similar equations for multiple-input/multiple-output
(MIMO) systems
Y( z )  H ( z ).U( z )
– Example : 2-inputs, 3 outputs
Y1 ( z )   H11 ( z ) H12 ( z ) 
Y ( z )   H ( z ) H ( z ). X 1 ( z ) 
22
 2   21
  X ( z )
Y3 ( z )   H 31 ( z ) H 32 ( z )   2 
DSP-II
Version 2009-2010
Lecture-2: Signals & Systems Review
p. 30
Discrete/Fast Fourier Transform 1/5
• DFT definition:
– the frequency spectrum/response of a discrete-time
signal/system x[k] is a (periodic) continuous function of
the radial frequency ω
X (e
j
 

)    x[k ]. z  k 
 k  
 z  e j
– The `Discrete Fourier Transform’ (DFT) is a discretized
version of this, obtained by sampling ω at N uniformly
spaced frequencies n  2n / N
(n=0,1,..,N-1)
and by truncating x[k] to N samples (k=0,1,..,N-1)
X (e
j
2n
N
N 1
)   x[k ]. e
j
2n
k
N
= DFT
k 0
DSP-II
Version 2009-2010
Lecture-2: Signals & Systems Review
p. 31
Discrete/Fast Fourier Transform 2/5
• Inverse discrete Fourier transform (IDFT):
– an -point DFT can be calculated from an
sequence:
-point time
= DFT
– vice versa, an -point time sequence can be calculated
from an -point DFT:
= IDFT
DSP-II
Version 2009-2010
Lecture-2: Signals & Systems Review
p. 32
Discrete/Fast Fourier Transform 3/5
• DFT/IDFT in matrix form
– Using shorthand notation..
– ..the DFT can be rewritten as
– an
DSP-II
-point DFT requires
Version 2009-2010
complex multiplications
Lecture-2: Signals & Systems Review
p. 33
Discrete/Fast Fourier Transform 4/5
• DFT/IDFT in matrix form
– Using shorthand notation..
– ..the IDFT can be rewritten as
– an
DSP-II
-point IDFT requires
Version 2009-2010
complex multiplications
Lecture-2: Signals & Systems Review
p. 34
Discrete/Fast Fourier Transform 5/5
•
•
•
•
•
•
•
•
•
split up N-point DFT in two N/2-point DFT’s
split up two N/2-point DFT’s in four N/4-point DFT’s
…
split up N/2 2-point DFT’s in N 1-point DFT’s
calculate N 1-point DFT’s
rebuild N/2 2-point DFT’s from N 1-point DFT’s
…
rebuild two N/2-point DFT’s from four N/4-point DFT’s
rebuild N-point DFT from two N/2-point DFT’s
– DFT complexity of
FFT complexity of
DSP-II
Version 2009-2010
multiplications is reduced to
John W.Tukey
Carl Friedrich Gauss (1777-1855
– divide-and-conquer approach:
James W. Cooley
• Fast Fourier Transform (FFT) (1805/1965)
multiplications
Lecture-2: Signals & Systems Review
p. 35
Need more?
• Introductory books
– S. J. Orfanidis, “Introduction to Signal Processing”, Prentice-Hall
Signal Processing Series, 798 p., 1996
– J. H. McClellan, R. W. Schafer, and M. A. Yoder, “DSP First: A
Multimedia Approach”, Prentice-Hall, 1998
– P. S. R. Diniz, E. A. B. da Silva and S. L. Netto, “Digital Signal
Processing: System Analysis and Design”, Cambridge University
Press, 612 p., 2002
• Online books
– Smith, J.O. Mathematics of the Discrete Fourier Transform (DFT),
http://ccrma.stanford.edu/~jos/mdft/, 2003, ISBN 0-9745607-0-7.
– Smith, J.O. Introduction to Digital Filters, August 2006 Edition,
http://ccrma.stanford.edu/~jos/filters06/.
DSP-II
Version 2009-2010
Lecture-2: Signals & Systems Review
p. 36