DSP-CIS
Chapter-2: Signals & Systems Review
Marc Moonen & Toon van Waterschoot
Dept. E.E./ESAT-STADIUS, KU Leuven
marc.moonen@esat.kuleuven.be
www.esat.kuleuven.be/stadius/
Chapter-2: Signals & Systems Review
• Introduction
Digital signal processing
• Discrete-Time/Digital Signals
(10 slides)
Sampling, quantization, reconstruction
• Discrete-Time Systems
(15 slides)
LTI, impulse response, convolution, z-transform, frequency
response, frequency spectrum, IIR/FIR
• Discrete/Fast Fourier Transform (5 slides)
DFT-IDFT, FFT
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
p. 2
Introduction: Digital Signal Processing?
• Signal = a physical quantity (e.g. voltage/current, intensity/grey-level,...)
that varies as a function of some independent variable(s),
(e.g. time, horizontal/vertical position, …)
– 1-dimensional: speech signal, audio signal,
electromagnetic/radio signal, …
– 2-dimensional: image, …
– N-dimensional
• Processing = `filtering’ (mostly) = noise reduction,
equalization, signal separation, …
• Digital = …in `digital domain’
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
p. 3
Analog signal processing
Analog Domain
(Continuous-Time Domain)
Analog
Signal
Processing
Circuit
Analog IN
Analog OUT
y (t )
u (t )

U ( f )   u (t ).e  j 2 . f .t dt
Jean-Baptiste Joseph Fourier (1768-1830)
Introduction: Digital Signal Processing?

Y( f ) 

 j 2 . f .t
y
(
t
).
e
dt


(=Spectrum/Fourier Transform)
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
p. 4
Introduction: Digital Signal Processing?
Digital signal processing in an analog world
Analog
domain
Analog IN
Analog-toDigital
Conversion
Digital
domain
011010
0101
100110
0010
DSP
Digital
IN
Digital
OUT
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
Analog
domain
Digital-toAnalog
Conversion
Analog OUT
p. 5
Discrete-Time/Digital Signals
Analog
domain
Analog IN
Digital
domain
Analog-toDigital
Conversion
011010
0101
100110
0010
DSP
Digital
IN
Digital
OUT
sampling
& quantization
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
1/10
Analog
domain
Digital-toAnalog
Conversion
Analog OUT
reconstruction
p. 6
Discrete-Time/Digital Signals 2/10 : Sampling
• Time-domain sampling
x(t )
amplitude
amplitude
continuous-time
signal
x[k ]  x(k .Ts )
discrete-time
signal
impulse
train
Ts
continuous-time (t)
01234
discrete-time [k]
It will turn out (page 27) that a spectrum can be
computed from x[k], which (remarkably) will be

equal to the spectrum (Fourier transform) of the xD (t )  x(t ).
 (t  k .Ts )
(continuous-time) sequence of impulses =
k  

DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
p. 7
Discrete-Time/Digital Signals 3/10 : Sampling
So what does this spectrum of xD(t) look like…
• Spectrum replication
– time domain:

xD (t )  x(t ).   (t  k .Ts )
k  
– frequency domain:
1 
k
XD( f )  .  X( f  )
Ts k 
Ts
X(f )
magnitude
frequency (f)
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
XD( f )
magnitude
frequency (f)
p. 8
Discrete-Time/Digital Signals 4/10 : Sampling
• Sampling theorem
– analog signal spectrum X(f) has a bandwidth of fmax Hz
– spectrum replicas are separated by fs =1/Ts Hz
magnitude
frequency
– no spectral overlap if and only if
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
p. 9
Discrete-Time/Digital Signals 5/10 : Sampling
– terminology:
• sampling frequency/rate fs
• Nyquist frequency fs/2
• sampling interval/period Ts
– e.g. CD audio: fs = 44,1 kHz
Harry Nyquist (1889 –1976)
• Sampling theorem
• Anti-aliasing prefilters
– if
then frequencies above the Nyquist
frequency are ‘folded’ into lower frequencies
(=aliasing)
– to avoid aliasing, sampling is usually preceded by
(analog-domain) low-pass (=anti-aliasing) filtering
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
p. 10
Discrete-Time/Digital Signals 6/10 : Quantization
• B-bit quantization
amplitude
discrete-time signal
x[k ]
quantized discrete-time signal
=discrete-amplitude/time signal
amplitude
=digital signal
xQ [k ]
3Q
2Q
Q
0
-Q
-2Q
-3Q
discrete time [k]
number of bits B = log2 (
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
R
discrete time [k]
range R
+1)
quantization step Q
p. 11
Discrete-Time/Digital Signals 7/10 : Quantization
• B-bit quantization:
– the quantization error
take on values between
can only
and
6dB per bit rule
– hence
can be considered as a random noise
signal (see below) with range
– the signal-to-noise ratio (SNR) of the B-bit quantizer can
then be defined as the ratio of the signal range and
the quantization noise range
:
– e.g. CD audio: 16bits -> 96dB SNR
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
(LP’s: 60dB SNR)
p. 12
Discrete-Time/Digital Signals 8/10 : Reconstruction
• Reconstructor =
– ‘fill the gaps’ between adjacent samples
– example: staircase reconstructor (see also next page) :
amplitude
x[k ]
amplitude xR (t )
discrete-time/digital
signal
discrete time [k]
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
reconstructed
analog signal
continuous time (t)
p. 13
Discrete-Time/Digital Signals 9/10 : Reconstruction
• xD(t) is generated first as an intermediate signal (after D-to-A &
sampler), which is then followed by (analog domain) filtering
XD( f )
• Ideal reconstructor =
magnitude
– ideal (rectangular) low-pass filter
– no distortion
frequency (f)
• Staircase reconstructor =
– D-to-A followed by `sample&hold’
– `hold’ corresponds to sinc-like
low-pass filter with sidelobes
– distortion due to
spurious high frequencies
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
XD( f )
magnitude
frequency (f)
p. 14
Discrete-Time/Digital Signals 10/10 : Reconstruction
• Anti-image post-filter
– low-pass (analog domain) filter succeeds reconstructor, to
remove spurious high frequency components due to
non-ideal reconstruction
• Complete scheme is…
Analog IN
Analog OUT
antialiasing
prefilter
antiimage
postfilter
sampler
quantizer
DSP
Digital
IN
reconstructor
Digital
OUT
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
p. 15
Discrete-Time Systems 1/15
Discrete-time (DT) system is `sampled data’ system
u[k]
y[k]
Input signal u[k] is a sequence of samples (=numbers)
..,u[-2],u[-1] ,u[0], u[1],u[2],…
System then produces a sequence of output samples y[k]
..,y[-2],y[-1] ,y[0], y[1],y[2],…
Example: `DSP’ block in previous slide
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
p. 16
Discrete-Time Systems 2/15
Will consider linear time-invariant (LTI) DT systems
u[k]
y[k]
Linear :
input u1[k] -> output y1[k]
input u2[k] -> output y2[k]
hence a.u1[k]+b.u2[k]-> a.y1[k]+b.y2[k]
Time-invariant (shift-invariant)
input u[k] -> output y[k],
hence input u[k-T] -> output y[k-T]
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
p. 17
Discrete-Time Systems 3/15
Causal systems
iff for all input signals with u[k]=0,k<0 -> output y[k]=0,k<0
Impulse response
input …,0,0, 1 ,0,0,0,...-> output …,0,0, h[0] ,h[1],h[2],h[3],...
General input u[0],u[1],u[2],u[3]
(cfr. linearity & shift-invariance!)
0
0
0 
 y[0] h[0]
 y[1]   h[1] h[0]
 u[0]
0
0


 

 y[2] h[2] h[1] h[0]
0   u[1] 


.
h[2] h[1] h[0] u[2]
 y[3]  0


 y[4]  0
0
h[2] h[1]  u[3]

 

0
0
h[2]
 y[5]  0
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
this is called a
`Toeplitz’ matrix
p. 18
Discrete-Time Systems 4/15
Convolution
u[0],u[1],u[2],u[3]
0
0 
 y[0] h[0] 0
 y[1]   h[1] h[0] 0
 u[0]
0


 

 y[2] h[2] h[1] h[0] 0   u[1] 



.
 y[3]  0 h[2] h[1] h[0] u[2]


 y[4]  0

0 h[2] h[1] u[3]

 

0
0 h[2]
 y[5]  0
y[0],y[1],...
h[0],h[1],h[2],0,0,...
D
y[k] = å h[k - k ].u[k ]= h[k]* u[k] = `convolution sum’
k
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
p. 19
Discrete-Time Systems 5/15
Z-Transform of system h[k] and signals u[k],y[k]
D
D
H (z)= å h[k].z
U(z)= å u[k].z
k
k
-k
-k
D
Y(z)= å y[k].z-k
k
0
0
0 
 y[0]
 h[0]
 y[1] 
 h[1] h[0]
0
0  u[0]



 y[ 2]
h[2] h[1] h[0]
0   u[1] 
1
2
3
4
5
1
2
3
4
5
1 z
z
z
z
z .
z
z
z
z .
 1 z
.
y
[
3
]
0
h
[
2
]
h
[
1
]
h
[
0
]



 u[ 2]


 y[ 4]
 0
0
h[ 2] h[1]  u[3]




0
0
h[ 2]
 y[5]
 0


 


Y ( z)


H ( z).1 z1
 Y ( z )  H ( z ).U ( z )





z2
z3





H(z) is `transfer function’
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
p. 20
Discrete-Time Systems 6/15
Z-Transform
• easy input-output relation: Y ( z )  H ( z ).U ( z )
• may be viewed as `shorthand’ notation
(for convolution operation/Toeplitz-vector product)
• stability
=bounded input u[k] leads to bounded output y[k]
--iff
 h[k ]  
k
--iff poles of H(z) inside the unit circle
(now z=complex variable)
(for causal,rational systems, see below)
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
p. 21
Discrete-Time Systems 7/15
Example-1 : `Delay operator’
u[k]

Impulse response is …,0,0 ,0, 1,0,0,0,…
Transfer function is
H (z) = z-1 =
Pole at z=0
1
z
Example-2 : Delay + feedback
Impulse response is …,0,0 ,0, 1,a,a^2,a^3…
Transfer function is
Pole at z=a
y[k]=u[k-1]
u[k]
+
H ( z )  z 1  a.z 2  a 2 .z 3  a 3 .z 4  ...
1
 H ( z )  a.z H ( z )  z
1
z 1
1
 H ( z) 

1  a.z 1 z  a
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
y[k]

x
a
=simple rational function
p. 22
Discrete-Time Systems 8/15
Will consider only rational transfer functions:
B(z) b0 zL + b1zL-1 +... + bL b0 + b1z-1 +... + bL z- L
H (z) =
= L
=
L-1
A(z)
z + a1z +... + aL
1+ a1z-1 +... + aL z-L
• In general, these represent `infinitely long impulse response’ (`IIR’)
systems
• L poles (zeros of A(z)) , L zeros (zeros of B(z))
• corresponds to difference equation
Y ( z )  H ( z ).U ( z )  A( z ).Y ( z )  B( z ).U ( z )  ...
y[k] = b0.u[k]+ b1.u[k -1]+... + bL .u[k- L]- a1.y[k -1]-... - aL.y[k - L]
• Hence rational H(z) can be realized with finite number of delay
elements, multipliers and adders
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
p. 23
Discrete-Time Systems 9/15
Special case is
B(z)
-1
-L
H (z) = L = b0 + b1z +... + bL z
z
• L poles at the origin z=0 (hence guaranteed stability)
• L zeros (zeros of B(z))
= `all zero’ filters
• corresponds to difference equation
Y(z) = H(z).U(z) Þ y[k] = b0.u[k]+ b1.u[k -1]+... + bL .u[k - L]
=`moving average’ (MA) filters
• impulse response h[k] is
0, 0, 0, b0 , b1,..., bL-1, bL , 0, 0, 0,...
= `finite impulse response’ (`FIR’) filters
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
p. 24
Discrete-Time Systems 10/15
Im
H(z) & frequency response:
u[2]
• given a system H(z)
• given an input signal = complex sinusoid
u[k ]  e jk

Re
Im
  k  
u[1]
u[0]=1
 cos(k )  j. sin( k )
• output signal :
Re
y[k] = å h[k ].u[k-k ] = å h[k ].ejw (k-k )=e jwk å h[k ].e- jwk = u[k].H(ejw )
k
k
j
H (e )
k
= `frequency response’
= complex function of radial frequency ω
= H(z) evaluated on the unit circle
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
p. 25
Discrete-Time Systems 11/15
H(z) & frequency response:
• periodic : period =
2
j
H (e )
• for a real-valued impulse response h[k]
- magnitude response H (e j )
is even function
- phase response
is odd function
j
H (e )
• example (`low pass filter’):
Nyquist frequency
1
e jk  ...,1,1,1,1,1,...
0.5
0
-4
5

-2
0
2

4

(=2 samples/period)
0
-5
-4
-2
0 Review 2/ Version 2013-2014
4
DSP-CIS / Chapter-2:
Signals
& Systems
p. 26
Discrete-Time Systems 12/15
• Z-transform & Fourier transform
– the frequency response H (e j ) of an LTI system (=z-transform
evaluated at the unit circle) is equal to the Fourier transform of the
continuous-time impulse sequence (see p.7) constructed with h[k]
F{hD (t)} = F{å h[k].d (t - k.Ts )} = ... = H(ejw ) , w = 2p .
k
jw
f
fs
jw
– the frequency spectrum U(e ) or Y(e ) of a discrete-time signal
(=z-transform evaluated at the unit circle) is equal to the Fourier
transform of the continuous-time impulse sequence constructed
with u[k] or y[k]
f
F{uD (t)} = F{åu[k].d (t - k.Ts )} = ... = U(ejw ) , w = 2p .
fs
k
• Input/output relation:
(from page 20)
Y (e j )  H (e j ).U (e j )
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
p. 27
Discrete-Time Systems 13/15
• Example: All-pass filter
– a (unity-gain) all-pass filter is a filter that passes all input
signal frequencies without gain or attenuation
H (e
j
) 1
– a (unity-gain) all-pass filter preserves signal energy
å u[k]
k
2
= å y[k]
2
k
– an all-pass filter may have any phase response
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
p. 28
Discrete-Time Systems 14/15
• Example: Biquadratic (=2nd order) all-pass filter
– it can be shown that for the unity-gain constraint to hold,
the denominator coefficients must equal the numerator
coefficients in reverse order, i.e.,
– the poles and zeros are then related as follows
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
p. 29
Discrete-Time Systems 15/15
• PS:
– So far have only considered single-input/single-output
(SISO) systems
Y ( z )  H ( z ).U ( z )
– Similar equations for multiple-input/multiple-output
(MIMO) systems
Y( z )  H ( z ).U( z )
– Example : 2-inputs, 3 outputs
Y1 ( z )   H11 ( z ) H12 ( z ) 
Y ( z )   H ( z ) H ( z ). X 1 ( z ) 
22
 2   21
  X ( z )
Y3 ( z )   H 31 ( z ) H 32 ( z )  2 
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
p. 30
Discrete/Fast Fourier Transform 1/5
• DFT definition:
– the frequency spectrum/response of a discrete-time
signal/system x[k] is a (periodic) continuous function of
the radial frequency ω (see p.27)
j
X (e
 

)    x[k ]. z  k 
 k  
 z  e j
– The `Discrete Fourier Transform’ (DFT) is a discretized
version of this, obtained by sampling ω at N uniformly
spaced frequencies wn = 2p .n / N
(n=0,1,..,N-1)
and by truncating x[k] to N samples (k=0,1,..,N-1)
j
X(e
2 p .n
N
N-1
-j
) = å x[k]. e
2 p .n
k
N
= DFT
k=0
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
p. 31
Discrete/Fast Fourier Transform 2/5
• DFT & Inverse DFT (IDFT):
– An N-point DFT sequence can be calculated from an
N-point time sequence:
= DFT
– Conversely, an N-point time sequence can be calculated
from an N-point DFT sequence:
= IDFT
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
p. 32
Discrete/Fast Fourier Transform 3/5
• DFT/IDFT in matrix form
– Using shorthand notation..
– ..the DFT can be rewritten as
– An N-point DFT requires
complex multiplications
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
p. 33
Discrete/Fast Fourier Transform 4/5
• DFT/IDFT in matrix form
– Using shorthand notation..
– ..the IDFT can be rewritten as
– An N-point IDFT requires
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
complex multiplications
p. 34
Discrete/Fast Fourier Transform 5/5
•
•
•
•
•
•
•
•
•
split up N-point DFT in two N/2-point DFT’s
split up two N/2-point DFT’s in four N/4-point DFT’s
…
split up N/2 2-point DFT’s in N 1-point DFT’s
calculate N 1-point DFT’s
rebuild N/2 2-point DFT’s from N 1-point DFT’s
…
rebuild two N/2-point DFT’s from four N/4-point DFT’s
rebuild N-point DFT from two N/2-point DFT’s
– DFT complexity of
multiplications is reduced to
FFT complexity of
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
John W.Tukey
Carl Friedrich Gauss (1777-1855
– divide-and-conquer approach:
James W. Cooley
• Fast Fourier Transform (FFT) (1805/1965)
multiplications
p. 35
Need more?
• Introductory books
– S. J. Orfanidis, “Introduction to Signal Processing”, Prentice-Hall
Signal Processing Series, 798 p., 1996
– J. H. McClellan, R. W. Schafer, and M. A. Yoder, “DSP First: A
Multimedia Approach”, Prentice-Hall, 1998
– P. S. R. Diniz, E. A. B. da Silva and S. L. Netto, “Digital Signal
Processing: System Analysis and Design”, Cambridge University
Press, 612 p., 2002
• Online books
– Smith, J.O. Mathematics of the Discrete Fourier Transform (DFT),
http://ccrma.stanford.edu/~jos/mdft/, 2003, ISBN 0-9745607-0-7.
– Smith, J.O. Introduction to Digital Filters, August 2006 Edition,
http://ccrma.stanford.edu/~jos/filters06/.
DSP-CIS / Chapter-2: Signals & Systems Review / Version 2013-2014
p. 36