1 Introduction
Backgrounds1
„ Signals, Systems, and Digital Signal Processing
Definition
Basic Elements of a Digital Signal Processing
Advantages of Digital over Analog Signal Processing
„ Classification of Signals
Multi-channel and Multi-dimensional Signals
Continuous-Time versus Discrete-Time Signals
Continuous-Valued versus Discrete-Valued Signals
„ Concepts of Frequency
Physical Interpretation of Signal Frequency
Continuous-Time Sinusoidal Signals
Discrete-Time Sinusoidal Signals
NCTU/CSIE/DSP LAB
Audio Processing Group
1.1 Signals, Systems, & Digital Signal
Processing
Backgrounds2
„ Definition
„ Basic Elements of DSP
„ Advantages of Digital over Analog Signal Processing
NCTU/CSIE/DSP LAB
Audio Processing Group
Definition
Backgrounds3
„ Signals
Any physical quantity that varies with time, space or
any other independent variable.
Communication beween humans and machines.
„ Systems
mathematically a transformation or an operator that
maps an input signal into an output signal.
can be either hardware or software.
such operations are usually referred as signal
processing.
„ Digital Signal Processing
The representation of signals by sequences of
numbers or symbols and the processing of these
sequences.
NCTU/CSIE/DSP LAB
Audio Processing Group
Basic Elements of a Digital Signal Processing
System
Backgrounds4
„ A/D Converter
Converts an analog
signal into a sequence
of digits
Analog
Input
Signal
Digital
Input
Signal
A/D
A/D
Converter
Converter
„ D/A Converter
Converts a sequence of
digits into an analog
signal
{3, 5, 4, 6 ...}
t
0
Digital
Digital
Signal
Signal
Processing
Processing
D/A
D/A
Converter
Converter
Analog
Output
Signal
Digital
Output
Signal
NCTU/CSIE/DSP LAB
Audio Processing Group
Advantages of Digital over Analog Processing
Backgrounds5
„ Better control of accuracy
„ Easily stored on magnetic media
„ Allow for more sophisticated signal
processing
„ Cheaper in some cases
NCTU/CSIE/DSP LAB
Audio Processing Group
1.2 Classification of Signals
Backgrounds6
„ Multichannel versus Multidimensional Signals
Signals may be generated by multiple sources or multiple sensors. Such
signals are multi-channel signals.
A signal which is a function of M independent variables is called
multidimensional signals.
„ Continuous-Time versus Discrete-Time Signals
Continuous-time signals are defined for every value of time.
Discrete -time signals are defined at discrete values of time.
„ Continuous-Valued versus Discrete-Valued Signals
A signal which takes on all possible values on a finite range or infinite
range is said to be a multi-channel signal.
A signal takes on values from a finite set of possible values is said to be a
multi-dimensionall signal.
NCTU/CSIE/DSP LAB
Audio Processing Group
Examples
Backgrounds7
„ A picture is a two-dimensional signal
I(x,y) is a function of two variables.
„ A black-and-white television picture is a three-dimensional
signal
I(x,y,t) is a function of three variables.
„ A color TV picture is a three-channel, three-dimensional
signals
Ir(x,y,t), Ig(x,y,t), and Ib(x,y,t)
NCTU/CSIE/DSP LAB
Audio Processing Group
1.3 Concepts of Frequency
Backgrounds8
„ Physical Interpretation of Signal
Frequency
„ Continuous-Time Sinusoidal Signals
„ Discrete-Time Sinusoidal Signals
NCTU/CSIE/DSP LAB
Audio Processing Group
Physical Interpretation of Signal
Frequency
Backgrounds9
„ A familiar term in physics and mathematics
Š
Š
Š
Radio transmitter/receiver
Amplifier
Color photography
...............
NCTU/CSIE/DSP LAB
Audio Processing Group
Physical Interpretation of Signal Frequency
Backgrounds10
„ Interpretation
Closed related to a specific type of periodic motion called harmonic
oscillation, described by sinusoidal functions.
Usually a dimension of inverse time.
Why is the term
important ?
Time
NCTU/CSIE/DSP LAB
Audio Processing Group
Physical Interpretation of Signal
Frequency
Time Domain
Representation
Backgrounds11
An Observation
Fourier Transform
Frequency-Domain
Representation
NCTU/CSIE/DSP LAB
Audio Processing Group
Physical Interpretation of Signal
Frequency
Backgrounds12
Signal can be represented either through time- or
frequency-domain.
Frequency-domain representation of signals provides
another viewpoint benefitial to signal analysis, human
sensitivity, system design, and phenomenon interpretation.
Frequency Transform: the tool to decompose a timedomain signal into frequency components.
The "frequency" can be considered as the varying rate of
the signal f(x) in x-domain.
f(t)
f(F)
Time
Spectrum
Frequency
NCTU/CSIE/DSP LAB
Audio Processing Group
1.3 Concepts of Frequency
Backgrounds13
„ Physical Interpretation of Signal Frequency
„ Continuous-Time Sinusoidal Signals
„ Discrete-Time Sinusoidal Signals
NCTU/CSIE/DSP LAB
Audio Processing Group
Continuous-Time Sinusoidal Signals
Backgrounds14
„ Definition
Xa(t) = A cos( Ω t+ θ)
-*<t<*
– A is the amplitude of the sinusoid
– Ω is the frequency in radians per∞
second
– θ is the phase in radians
– F=Ω/2π is the frequency in
cycles per second or hertz
Time
NCTU/CSIE/DSP LAB
Audio Processing Group
Continuous-Time Sinusoidal Signals (Cont.)
Backgrounds15
„ For every fixed value of F, Xa(t) is periodic
Xa(t+Tp) = Xa(t), Tp=1/F
„ Continuous-time sinusoidal signals with distinct
frequencies are themselves distinct
„ Increasing the frequency F results in an increase in the
rate of oscillation
NCTU/CSIE/DSP LAB
Audio Processing Group
Discrete-Time Sinusoidal Signals
Backgrounds16
„ Definition
X(n) = A cos( ω n+ θ), -*<t<*
– A is the amplitude of the sinusoid
– ω is the frequency in radians per second
– θ is the phase in radians
– f=ω/2π is the frequency in cycles per
second or hertz
X(n) = A cos( ω n+ θ)
NCTU/CSIE/DSP LAB
Audio Processing Group
Discrete-Time Sinusoidal
Signals(Cont.)
„ A discrete-time sinusoidal is periodic
only if its frequency f is a rational
number
Backgrounds17
X(n) = A cos( ω n+ θ)
– X(n+N) = X(n), N=p/f, where p is an
integer
„ Discrete-time sinusoidal signals where
frequencies are separated by an integer
multiple of 2π are identical
– X1(n) = A cos( ω0 n)
– X2(n) = A cos( (ω0 +2π) n)
„ The highest rate of oscillation in a
discrete-time sinusoidal is attained
when ω=π or (ω=-π), or equivalently
f=1/2.
– X(n) = A cos(( ω0+π)n) = -A cos((ω0+π)n
NCTU/CSIE/DSP LAB
Audio Processing Group
2. The Process of A/D and D/A Conversion
Backgrounds18
„
„
„
„
„
„
Basic Elements
Signal Sampling
Anti-aliasing Filtering
Quantization
Interpolator
Smoothing Filters
NCTU/CSIE/DSP LAB
Audio Processing Group
2.1 Basic Elements
Analog
Input
Signal
Backgrounds19
Sampling
Frequency Fs
Cut-off
Frequency Fc
Antialiasing
Antialiasing
Filter
Filter
Sampler
Sampler
The number
of bits
Quantizer
Quantizer
t
{3, 5, 4, 6 ...}
0
t
t
t
t
Smoothing
Smoothing
Filter
Filter
Analog
Output
Signal
Cut-off
Frequency Fc*
Digital
Digital
Signal
Signal
Processing
Processing
{3, 5, 4, 6 ...}
Interpolator
Interpolator
Sampling
Frequency Fs*
NCTU/CSIE/DSP LAB
Audio Processing Group
2.1 Basic Elements(c.1)
Backgrounds20
„ An Observation
The mapping between discrete-frequency and analogfrequency is one-to many
NCTU/CSIE/DSP LAB
Audio Processing Group
2.1 Basic Elements(c.2)
Backgrounds21
„ Sampler
Converts a continuous-time signal into a discrete-time signal.
„ Anti-aliasing Filter(A low-pass filter)
Deletes the frequency components above a threshold frequency to avoid
the aliasing effects.
„ Quantizer
Converts a discrete-time continuous-valued signal into a discrete-time,
discrete-valued signal
Antialiasing
Antialiasing
Filter
Filter
t
0
2.1 Basic Elements
Fc
Sampler
Sampler
Fs
t
Quantizer
Quantizer
{3, 5, 4, 6 ...}
t
NCTU/CSIE/DSP LAB
Audio Processing Group
2.1 Basic Elements(c.3)
Backgrounds22
„ Interpolator
Converts a discrete-time signal into a continuous-time signal.
„ Smoothing Filter
Deletes the frequency components above a threshold frequency to avoid
the image signal.
{3, 5, 4, 6 ...}
t
t
t
Smoothing
Smoothing
Filter
Filter
Analog
Output
Signal
Cut-off
Frequency
Fc*
Interpolator
Interpolator
Sampling
Frequency Fs*
NCTU/CSIE/DSP LAB
Audio Processing Group
2.1 Basic Elements(c.4)
Analog
Input
Signal
Sampling
Frequency Fs
Cut-off
Frequency Fc
Antialiasing
Antialiasing
Filter
Filter
Backgrounds23
Sampler
Sampler
The number
of bits
Quantizer
Quantizer
&&Coder
Coder
t
0
t
t
t
{3, 5, 4, 6 ...}
t
Smoothing
Smoothing
Filter
Filter
Analog
Output
Signal
Cut-off
Frequency
Fc*
{3, 5, 4, 6 ...}
Digital
Digital
Signal
Signal
Processing
Processing
Interpolator
Interpolator
Sampling
Frequency Fs*
NCTU/CSIE/DSP LAB
Audio Processing Group
2.2 Signal Sampling
Backgrounds24
‹Sampling
• The conversion of a continuous-time signal into a discrete-time signal
obtained by taking "samples" of the continuous-time signal at
discrete-time instants
Xa(nT) = X(n)
where T is the sampling interval
Many-to-One Mapping
between F and f
Time
X(t)
X(n)
NCTU/CSIE/DSP LAB
Audio Processing Group
2.2 Signal Sampling (c.1)
Analog Frequency <==> Discrete FrequencyBackgrounds25
‹ The relationship between the time variables t and n
• t = nT = n/Fs
‹ Analog Frequency F (or Ω) <==> Discrete Frequency f (ω)
• Xa(nT) = x(n) = Acos(2πFnT +θ) = A cos (2pnF/Fs + θ)
• compare with x(n) = A cos (2πfn+θ)
• f = F/Fs or ω = ΩT
F
f = F/Fs
or ω = ΩT
f
X(n) = A cos( ω n+ θ)
NCTU/CSIE/DSP LAB
Audio Processing Group
2.2 Signal Sampling (c.2)
Frequency Restriction
‹Continuous-Time Frequency
-∗<F<
-*<Ω<
‹ Discrete-Time Frequency
- 1/2 < f < 1/2
-π<ω<π
Backgrounds26
Many-to-One Mappling
between F and f
‰ Relation and Restriction
- Fs/2 < F < Fs/2
- πFs < Ω < πFs
NCTU/CSIE/DSP LAB
Audio Processing Group
2.2 Signal Sampling (c.3)
Frequency Relation
Backgrounds27
‹ Many-to-one Mapping
Fk = F0 + kFs are indistinguishable from the frequency F0 after
resampling and hence they are aliased of F0.
f
0
-Fs -Fs/2 Fs/2 Fs
F
‹ Folding Frequency ==> Fs/2
NCTU/CSIE/DSP LAB
Audio Processing Group
2.2 Signal Sampling (c.4)
Backgrounds28
‹Sampling Theorem
• If the highest frequency contained in an analog signal Xa(t) is
Fmax =B and the signal is sampled at a rate Fs > 2Fmax = B,
then Xa(t) can be exactly recovered from its sample values
using the interpolation
sin 2 π Bt
g(t ) =
2 π Bt
Thus Xa(t) may be expressed as
∞
n
Xa ( n / Fs ) g( t − )
x a (t ) = n∑
Fs
=−∞
where Xa(n/Fs) = Xa(nT) = X(n) are the sample of Xa(t)
NCTU/CSIE/DSP LAB
Audio Processing Group
2.2 Signal Sampling (c.5)
Backgrounds29
„ History
Cauchy, French, 1841
– Functions could be nonuniformly sampled and averaged over a long
period.
Whittaker, Scottish, 1915
– A bandlimited function can be completely reconstructed from
samples. (first mathematical proof of a general sampling theorem)
K. Ogura, Japanese, 1920
– If a function is sampled at a frequency at least twice the highest
function frequency, the samples contain all the information in the
function, and can reconstruct the function.
Carson, American, 1920
– Unpublished proof that related the same result to communication.
NCTU/CSIE/DSP LAB
Audio Processing Group
2.2 Signal Sampling (c.6)
Backgrounds30
„ History (c.1)
Nyquist, Sweden, 1928
– For complete signal reconstruction, the required frequency bandwidth
is proportional to the signalling speed.
– The minimum bandwidth is equal to half the number of code elements
per second.
– Expressed the theorem in terms that are familiar to communication
engineers.
Kotelnikov, Russian, 1933
– A proof of sampling theorem
Shannon, American, 1949
– Unified many aspects of sampling and founded the larger science of
information theory.
NCTU/CSIE/DSP LAB
Audio Processing Group
2.3 Antialiasing Filters
Backgrounds31
„ Aliasing
F +- kFs are mapped into the
same discrete frequency
f
-Fs -Fs/2
F
0
Fs/2
Fs
NCTU/CSIE/DSP LAB
Audio Processing Group
2.3 Antialiasing Filters
1
 Purpose:
ÎDelete the frequency components that
will be aliased to low frequency
components.
Low-Pass Filter
Fc
Backgrounds32
F
ÎLow-Pass Filters
ÎFc < Fs/2
NCTU/CSIE/DSP LAB
Audio Processing Group
2.4 Quantization
z Quantization
Backgrounds33
Output of Sampler
Î Express each sample value as a finite
number of digits.
z Quantization Error
Î The error introduced in representing
the continuous-value signal by a
discrete value levels.
Output of Quantization
z Signal-to-quantization noise
ratio, SQNR(dB)
Î 1.76 + 6.02b
Î 16 bits CD audio data has a quality of
more than 96 dB
NCTU/CSIE/DSP LAB
Audio Processing Group
2.4 Quantization (c.1)
Backgrounds34
„ SQNR(dB)
The maximum root mean
square signal Srms is
S rms =
Q 2 b −1
21 / 2
The rms quantization error is
E rms =  ∫ e 2 p ( e ) de 
 −∞

∞
1/ 2
1
=
Q
∫
∞

1/ 2
e 2de 
−∞

Q 2 
= 
 12 
1/ 2
=
Q
( 12 )1 / 2
The poweer ratio is
S
3

( dB ) = 10 log  ( 2 2b )  = 6 . 02b + 1. 76
E
2

NCTU/CSIE/DSP LAB
Audio Processing Group
2.4 Quantization (c.2)
Backgrounds35
„ Observation
The quantization error is random
and perceptually similar to analog
white noise for large amplitude
signals.
Problems
– low-amplitude signals.
– narrow band signals.
„ Dither
Decorrelates the errors from the
signals.
Allows the digital system to
encode amplitude smaller than the
LSB.
NCTU/CSIE/DSP LAB
Audio Processing Group
2.4 Quantization-- Types of Dither
Backgrounds36
„ Gaussian pdf
Contributes to Q2/4
„ Rectangular pdf
Contributes to Q2/12
„ Triangular pdf
Contributes to Q2/6
NCTU/CSIE/DSP LAB
Audio Processing Group
2.4 Quantization (c.4)
Backgrounds37
„ Earliest Dither in Word War II
Jim MacArthur has pointed out
Bombers used mechanical
computers to perform navigation
and bomb trajectory calculations.
These computers perform more
accurately when flying on board
the aircraft and less well on
ground.
Engineers realized that the
vibration from the aircraft reduced
the error from sticky moving
parts.
NCTU/CSIE/DSP LAB
Audio Processing Group
Optimal Interpolator
2.5 Interpolator
Backgrounds38
„ Optimal Interpolator:
• from Sampling Theorems
∞
n
Xa (n / Fs)g(t − )
xa (t ) = n∑
Fs
=−∞
Zero-order
Interpolator
• no distortion for the frequency
components below Fs/2
• no frequency components above Fs/2
exist and smoothing filtering is not
necessary
First-order
Interpolator
„ Suboptimal Interpolator
Signal Mangitude Spectrum
F
Fs
2Fs
• distortion exists for the frequency
components below Fs/2
• result in passing frequencies above
the folding frequency and smoothing
filtering is necessary
NCTU/CSIE/DSP LAB
Audio Processing Group
2.6 Smoothing Filters
Backgrounds39
„ Delete the frequency components above a threshold
frequency to avoid the image signal introduced by
suboptimal filters
Low-pass filtering
Low-Pass Filter
1
Fc'
Smoothing
Smoothing
Filter
Filter
F
t
t
Cut-off
Frequency Fc*
Signal Mangitude Spectrum
F
2Fs
0
Fs
2Fs
NCTU/CSIE/DSP LAB
Audio Processing Group
2.7 Concluding Remarks
Backgrounds40
„ Time/Frequency Illustrarion
Antialiasing filtering and Antiimaging filtering
NCTU/CSIE/DSP LAB
Audio Processing Group
2.7 Concluding Remarks
Cut-off
Frequency Fc
Analog
Input
Signal
Antialiasing
Antialiasing
Filter
Filter
Sampling
Frequency Fs
Sampler
Sampler
Backgrounds41
The number
of bits
Quantizer
Quantizer
t
{3, 5, 4, 6 ...}
0
t
t
t
t
Analog
Output
Signal
Smoothing
Smoothing
Filter
Filter
Cut-off
Frequency Fc*
{3, 5, 4, 6 ...}
Digital
Digital
Signal
Signal
Processing
Processing
Interpolator
Interpolator
Sampling
Frequency Fs*
NCTU/CSIE/DSP LAB
Audio Processing Group
Experiment 1
Backgrounds42
WinAmp Architectures
Describe the functionality of Input Plugin, Output Plugin, DSP Plugin, and
VIS Plugin.
„ Find the Input Plugin for Wave File.
„ Change the decoded results for Stereo Channels as
L′[n] = αL[n] + β {L[n] − R[n]}
R′[n] = αR[n] + β {R[n] − L[n]}
„ Find the suitable parameters for the two parameters.
„ Describe the noise you have found during the
experiments.
NCTU/CSIE/DSP LAB
Audio Processing Group
Experiment 2
Backgrounds43
„ Sampling Rates Change Problem.
1. Change the sampling rates from 44.1kHz to 22.05 kHz by eliminate the odd
samples of stereo channels.
L’[n] = L[2n]
R’[n] = R[2n]
– Listen to the resulted music and describe the artifacts.
– Compare the spectrum through COOL editor to find the spectrum
artifact.
2. Change again the sampling rates from 44.1 kHz to 11.025 kHz by three
samples every four samples.
L’[n] = L[4n]
R’[n] = R[4n]
– Listen to the resulted music and describe the artifacts.
– Compare the spectrum through COOL editor to find the spectrum
artifact.
NCTU/CSIE/DSP LAB
Audio Processing Group
Experiments
Analog
Input
Signal
Cut-off
Frequency Fc
Antialiasing
Antialiasing
Filte
r
Filter
t
0
„
„
„
„
„
Backgrounds44
Sampling
Frequency Fs
Sampler
Sampler
The number
of bits
Quantizer
Quantizer
Sampling
Frequency Fs*
t
Smoothing
Smoothing
Filter
Filter
Interpolator
Interpolator
{3, 5, 4, 6 ...}
t
Fc=Fc' and is below 1.5 k ==> Lowpass filtering effects
Fc > Fs/2 ==> Aliasing effects
Quantization effects
Fs' > Fs or Fs' < Fs ==> Frequency mismatching
Fc' > Fs/2 ==> Image effects
Cut-off
Frequency
Fc*
t
t
Analog
Output
Signal
NCTU/CSIE/DSP LAB
Audio Processing Group
Hearing Area in Frequency Domain
Backgrounds45
1. Blind Deconvolution for the first music
2. Piano Music
a. Original One
b. Low-pass One
c. Image Distortation (too many high frequency)
d. Aliasing effects
Quantization Noise is independent of the Original Signals ?
NCTU/CSIE/DSP LAB
Audio Processing Group