Chapter 9
Adaptive Filters
Objectives
• Show and describe the structure of the discrete Wiener
filter.
• Derive the statistical requirement for optimal estimation
of a signal component using the discrete Wiener filter.
• Describe and demonstrate the basic operation of the
Widrow-Hoff least-mean-squares (LMS) algorithm.
• Describe and configure an adaptive predictor and
demonstrate its use in the minimization of interfering
noise.
• Describe the principles of adaptive system identification
and demonstrate its use in several applications.
The Discrete Wiener Filter
+
Σ

-
y[k] = s[k] +n[k]

ek  y[k ] n[k ]  s[k ]
(signal + noise)
(output = signal estimate)
Wiener Filter
x[k]
(noise)

N 1
n[k ]   w[i ]x[ k  i]
i 0
(noise estimate)
The Adaptive Filter Structure
Error Signal
Feedback
Adaptive
Algorithm
The Least-Mean-Squares Adaptive
Algorithm (Widrow-Hoff Algorithm)
•
The adaptive algorithm acts as a “negative feedback” to minimize the error
signal by adjusting the coefficients of the Wiener filter. The signal is
delivered to the filter sample-by-sample. For the kth sample:
y[k ]  s[k ]  n[k ]

Uncorrelated signal components
N 1
n[k ]   wk [i]x[k  i]
Estimate of n[k] from the Wiener filter
i 0


ek  y[k ]  n[k ]  s[k ]
Error signal at the kth sample
wk 1[i]  wk [i]  2ek x[k  i]
wk 1[i]   wk [i]  2ek x[k  i]
Update the filter coefficients
The LMS Error Surface for a
Wiener Filter
M-File for the Widrow-Hoff Algorithm
(help file)
>> help adapt2
Least-Mean-Squares Active Adaptive Filter Routine
[err,n_hat,W] = ADAPT2(len,x,y,mu,gamma)
This is a general Widrow-Hoff LMS adaptive routine. A signal (X) is applied to
a filter whose coefficients are adjusted by LMS. The output (N_HAT) of
this filter is subtracted from an input signal (Y), producing an
error signal (ERR). This error signal is fed-back to the adaptive filter
to adjust the coefficients. The LMS routine minimizes the error signal power.
The final coefficients of the filter are given by the vector W.
LEN: the desired FIR filter length
X: a vector of input values to the adaptive Wiener filter.
Y: a vector of the signal to be compared to the output of the adaptive filter.
This vector must be the same length as the filtered input vector.
MU: LMS constant
GAMMA: "nudge-to-zero" constant for controlling fluctuations of the filter
coefficients. If specified, gamma should be close to but less than 1.
If the mu and gamma values are not specified, they are assumed to be 0.025 and 1
respectively.
ERR: err = Y - n_hat where n_hat is the output of the FIR filter.
N_HAT: the output of the filter when the X is applied.
W: a column vector describing the weights (coefficients) of the FIR filter.
The Adaptive Predictor
y[k]=s[k] + n[k]
(signal + uncorrelated noise)
+
Σ
-

ek  n[k ]
(output = uncorrelated
signal estimate)
Delay
Wiener Filter

N 1
s[k ]   w[i ]x[k  i ]
i 0
(correlated signal
estimate)
The Adaptive Predictor is typically used to “de-noise” signals.
The de-noised signal is the output of the Wiener filter.
Noise Removal From Sinusoidal
Signals
>> t=0:.001:.59;
% The signal will be one second long with a 1 ms
sampling period
>> x=sin(2*pi*20*t);
>> xn=x+.5*randn(size(t)); % This adds Gaussian noise to the
sinusoid
>> h=[zeros(1,20),1]; % This is the impulse response of a 20 step
% delay filter
>> xnd=filter(h,1,xn);
% This gives a 20 step delayed copy of the
% signal xn
>> [err,n_hat,w]=adapt2(40,xnd,xn,.001,1);
>> subplot(3,1,1), plot(xn); title('Noisy Signal')
>> subplot(3,1,2), plot(n_hat); title('Filter Output')
>> subplot(3,1,3), plot(err); title('Error Signal')
Noise Removal From Sinusoidal
Signals
Noisy Signal
5
0
-5
0
100
200
300
400
500
600
400
500
600
400
500
600
Filter Output
2
0
-2
0
100
200
300
Error Signal
2
0
-2
0
100
200
300
Noise Removal From Sinusoidal
Signals
>> fvtool(w,1)
% The final Wiener filter coefficients are in the vector w
Magnitude Response
0.9
0.8
0.7
0.6
Magnitude
The Wiener
filter converged
to a narrow
band-pass
centered on 20
Hz, thus
isolating that
component of
the signal
0.5
0.4
0.3
0.2
0.1
0
0
0.05
0.1
0.15
0.2
0.25
0.3
Frequency (kHz)
0.35
0.4
0.45
Noise Removal From Sinusoidal
Signals (3 Frequencies)
>> t=0:.001:.59;
>> x3=sin(2*pi*10*t)+sin(2*pi*20*t)+sin(2*pi*30*t);
>> x3n=x3+0.5*randn(size(x3));
>> h=[zeros(1,20),1];
>> x3nd=filter(h,1,x3n);
>> [err,n_hat,w]=adapt2(100,x3nd,x3n,.001,1);
>> subplot(4,1,1), plot(x3); title('Pure Signal')
>> subplot(4,1,2), plot(x3n); title('Noisy Signal')
>> subplot(4,1,3), plot(n_hat); title('Filter Output')
>> subplot(4,1,4), plot(err); title('Error Signal')
Pure Signal
5
0
-5
0
100
200
300
Noisy Signal
400
500
600
0
100
200
300
Filter Output
400
500
600
0
100
200
300
Error Signal
400
500
600
0
100
200
300
400
500
600
5
0
The Wiener filter has
converged after about
150 samples. The
filter is longer (order
100) in this case
because of the close
spacing of the
frequency
components
-5
5
0
-5
5
0
-5
Noise Removal From Sinusoidal
Signals (3 Frequencies)
>> fvtool(w,1)
Magnitude Response
1.4
1.2
1
Magnitude
The Wiener
filter has
converged to 3
band-pass
filters at the
frequencies of
the sinusoids
0.8
0.6
0.4
0.2
0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Normalized Frequency ( rad/sample)
0.8
0.9
Adaptive System Identification
x[k]
+
Σ
Unknown System
y[k]
-

ek  n[k ]
(output = residual noise
for matched systems)
Wiener Filter

N 1
y[k ]   w[i ]x[k  i ]
i 0
Estimate of
unknown system
Since we are trying to match the frequency response of the
unknown system, the input signal x[k] is Gaussian noise, because
its power is uniformly distributed in the frequency domain.
Example – Identifying an Impulse
Response
>> n=0:40;
% The length of the unknown system is 41.
>> omega=pi/4; % This sets the cut-off frequency of the low-pass filter
>> h=(omega/pi)*sinc(omega*(n-20)/pi); % This computes the impulse response
>> N=0:200; % This sets the signal length to 201 samples
>> x=randn(size(N)); % The signal x is pure Gaussian noise.
>> y=filter(h,1,x);
>> [err,n_hat,w]=adapt2(41,x,y,.015,1);
>> subplot(2,1,1),stem(h);title('Unknown System')
Unknown System
>> subplot(2,1,2),stem(w);title('Estimated System')
0.3
0.2
0.1
The adaptive filter
has matched the
“unknown” filter
impulse response
almost perfectly
0
-0.1
0
5
10
15
20
25
30
35
40
45
30
35
40
45
Estimated System
0.3
0.2
0.1
0
-0.1
0
5
10
15
20
25
Example – Sonar/Radar Ranging
+
x[k]
(transmitted signal)
Σ
Echo Delay
Received Signal
(Echo)
2X noise
Wiener Filter

N 1
y[k ]   w[i ]x[k  i ]
i 0
Estimate of echo delay
(w coefficients)
error
Σ
-
Example – Sonar/Radar Ranging
>> hd=[zeros(1,60),1]; % This the impulse response of a 60 step delay filter
>> x=[4*ones(1,8),zeros(1,192)];
>> xd=filter(hd,1,x); % This is the “ping” delayed by 60 steps
>> echo=xd+2*randn(size(xd)); % The delayed “ping” is contaminated with noise
>> [err,n_hat,w]=adapt2(100,x,echo,.001,1);
>> subplot(4,1,1),plot(x);axis([0 200 -10 10]);title('Transmitted Signal')
>> subplot(4,1,2),plot(echo);axis([0 200 -10 10]);title('Received Echo Signal')
>> subplot(4,1,3),stem(hd);axis([0 100 -1 1]);title('Echo Delay Impulse Response')
>> subplot(4,1,4),stem(w);axis([0 100 -.35 .35]);title('Estimated Delay')
Transmitted Signal
10
0
-10
Although the estimated
delay (impulse response)
is quite noisy, the
maximum value is at 60
samples, the delay of the
reflected echo signal.
0
20
40
60
80
100
120
Received Echo Signal
140
160
180
200
0
20
40
60
80
100
120
140
Echo Delay Impulse Response
160
180
200
0
10
20
30
40
50
60
Estimated Delay
70
80
90
100
0
10
20
30
40
70
80
90
100
10
0
-10
1
0
-1
0.2
0
-0.2
50
60
Example – FIR Filters with an IIR
Frequency Response
• An interesting example of system identification is using
the Wiener filter (FIR) to match the frequency response
of an IIR filter
• This is using the adaptive filter to solve an otherwise
difficult design problem.
• To illustrate the procedure, let’s first use MATLAB design
tools to design an IIR filter with a low-pass Butterworth
response to the following specifications:
–
–
–
–
Sampling frequency = 2000 Hz
Transition from the pass-band to the stop-band: 300 to 400 Hz
Pass-band variation less than 1 dB
Stop-band attenuation at least -20 dB
Example – FIR Filters with an IIR
Frequency Response
>> [N1,Wn1]=buttord(.3,.4,1,20);
>> [B1,A1]=butter(N1,Wn1);
>> length(B1)
ans =
10
>> length(A1)
ans =
10
Design a Butterworth filter
• Since this IIR filter has a total of 20 coefficients, it is reasonable that
an FIR realization of the same filter must have more coefficients; for
example, 40 coefficients
Example – FIR Filters with an IIR
Frequency Response
>> n=0:1000;
% Make the signals approximately 1000 samples long.
>> x=randn(size(n));
>> y=filter(B1,A1,x);
>> [err,n_hat,w]=adapt2(40,x,y,.01,1); % Make the Wiener filter length 40. Mu and gamma values are found
experimentally to result in a small error signal
>> fvtool(B1,A1)
>> fvtool(w,1)
Magnitude Response
1.4
1.2
1.2
1
1
0.8
Magnitude
Magnitude
Magnitude Response
1.4
0.6
0.8
0.6
0.4
0.4
0.2
0.2
0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Normalized Frequency ( rad/sample)
0.8
Butterworth filter response
0.9
0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Normalized Frequency ( rad/sample)
Adaptive filter response
0.8
0.9
Summary
• Adaptive filters respond in real-time to statistical
properties of signals
• Many adaptive filters are based on the discrete
Wiener filter and the Widrow-Hoff least-meansquares algortithm.
• Several applications were illustrated:
– Adaptive predictor: removal of noise or interfering
tones from signals
– System identification: sonar/radar ranging and design
of FIR filters with a classic IIR response.