DEPARTMENT OF INFORMATICS
March 2003
David Gesbert
Signal and Image Processing Group (DSB)
http://www.ifi.uio.no/~gesbert
D. Gesbert: IN357 Statistical Signal Processing1 of 17
Course book: Chap. 7.2 Statistical Digital Signal Processing and
modeling, M. Hayes 1996.
IN357: WIENER FILTERING
UNIVERSITY
OF OSLO
DEPARTMENT OF INFORMATICS
D. Gesbert: IN357 Statistical Signal Processing2 of 17
• Application 3: Multi-antennas diversity combining
• Application 2: Noise canceling
• Application 1: Linear prediction
• Finding the solution: The Wiener-Hopf equations
• Geometrical interpretation
• Optimum linear filtering
• Example 1: temporal Wiener filtering
• Purpose of Wiener filtering
Outline
UNIVERSITY
OF OSLO
..
DEPARTMENT OF INFORMATICS
p−1
x (n)
x 0(n)
x2 (n)
desired signal
d(n)
W
filter
error signal
e(n)
estimated signal
D. Gesbert: IN357 Statistical Signal Processing3 of 17
^
d(n)
observed random process
desired random process (unobserved)
observed random process
observed random process
p observations
{xp−1(n)}
{d(n)}
{x0(n)}
{x2(n)}
Goal: “To estimate an unknown random process (RP) from a set of
observed RPs, with which it is correlated.”
Purpose of Wiener filtering
UNIVERSITY
OF OSLO
DEPARTMENT OF INFORMATICS
D. Gesbert: IN357 Statistical Signal Processing4 of 17
• (2) one uses knowledge of correlation between {xi(n)} and d(n).
• (1) one has a training signal for d(n) and one adjusts W to minimize
the power of e(n).
Two solutions to find the operator (filter) W :
Purpose of Wiener filtering (II)
UNIVERSITY
OF OSLO
DEPARTMENT OF INFORMATICS
D. Gesbert: IN357 Statistical Signal Processing5 of 17
xi(n) = x(n − i), i = 0.., p − 1
W (z) = wo + w1z −1 + w1z −2 + .. + wp−1z −p+1
..
ˆ
d(n)
= w0x(n) + w1x(n − 1) + w2x(n − 2) + ... + wp−1x(n − p + 1)
⇒ {xi(n)}, i = 0, ..p − 1 are the samples of a wide sense stationary (WSS)
random signal.
⇒ W is a linear FIR filter.
Example 1: Linear temporal filtering
UNIVERSITY
OF OSLO
DEPARTMENT OF INFORMATICS
where T is the transpose operator.
ˆ
d(n)
= W T X(n)
D. Gesbert: IN357 Statistical Signal Processing6 of 17
W = [w0, w1, .., wp−1]T
X(n) = [x0(n), x1(n), .., xp−1(n)]T
Vector Formulation
UNIVERSITY
OF OSLO
DEPARTMENT OF INFORMATICS
D. Gesbert: IN357 Statistical Signal Processing7 of 17
Find Wo such that J(Wo) is minimum. Wo is the optimum linear filter in
the Wiener sense.
where E() is the expectation.
ˆ
e(n) = d(n) − d(n)
J(W ) = E|e(n)|2
Problem: One wishes to find the linear filter W that minimizes the error
ˆ
between d(n) and d(n).
Optimum linear filtering
UNIVERSITY
OF OSLO
DEPARTMENT OF INFORMATICS
x0 (n)
1
x (n)
• Orthogonality = decorrelation
d(n)
^
d(n)
e(n)
D. Gesbert: IN357 Statistical Signal Processing8 of 17
p=2
• The scalar product between two RPs {u(n)} and {v(n)} is defined by
the correlation: < u, v >= E(u(n)v(n)∗).
• WSS random processes ({e(n)}, {d(n)}, {x0(n)}, .., {xp−1(n)} can be
viewed as elements in a vector space.
Vector space representation
Geometrical Interpretation
UNIVERSITY
OF OSLO
DEPARTMENT OF INFORMATICS
(2)
(3)
(1)
D. Gesbert: IN357 Statistical Signal Processing9 of 17
Rx = E(X(n)∗X(n)T )
rdx = E(d(n)X(n)∗)
RxWo = rdx where
The solution Wo is given by the Wiener-Hopf equations.
Finding the solution
UNIVERSITY
OF OSLO
DEPARTMENT OF INFORMATICS







o
∂J
∗
∂wp−1
|W =W
..
∂J
∂w0∗ |W =Wo
∂J
∂w1∗ |W =Wo
(4)
D. Gesbert: IN357 Statistical Signal Processing10 of 17
 
0

 0
  
=.
 .

0

Direct method using optimization theory: The extrema of the cost
function J(W ) are found from (see book by Hayes p.49):
Derivation of the Wiener-Hopf equation
UNIVERSITY
OF OSLO
DEPARTMENT OF INFORMATICS
x0 (n)
1
x (n)
d(n)
^
d(n)
e(n)
(5)
D. Gesbert: IN357 Statistical Signal Processing11 of 17
p=2
< e(n), x0(n) >= E(e(n)x0(n)∗) = 0
< e(n), x1(n) >= E(e(n)x1(n)∗) = 0
..
< e(n), xp−1(n) >= E(e(n)xp−1(n)∗) = 0
• {e(n)} is orthogonal to the space spanned by {x0(n)}, .., {xp−1(n)}.
ˆ
• {d(n)}
is the projection of {d(n)} onto the space spanned by
{x0(n)}, .., {xp−1(n)}, or equivalently:
ˆ
{d(n)}
is the optimal estimate iff
Derivation using geometry
UNIVERSITY
OF OSLO
DEPARTMENT OF INFORMATICS
D. Gesbert: IN357 Statistical Signal Processing12 of 17
H
Wo
J(Wo) = E|d(n)|2 + WoH RxWo − WoH rdx − rdx
H
J(Wo) = E|d(n)|2 − rdx
Wo Wo
The minimum error, obtained with Wo, is J(Wo):
Expression for the minimum error
UNIVERSITY
OF OSLO
DEPARTMENT OF INFORMATICS
Hayes’ book, p. 342
D. Gesbert: IN357 Statistical Signal Processing13 of 17
Application 1: Linear prediction
UNIVERSITY
OF OSLO
DEPARTMENT OF INFORMATICS
Hayes’ book, p. 349
D. Gesbert: IN357 Statistical Signal Processing14 of 17
Application 2: Noise canceling
UNIVERSITY
OF OSLO
DEPARTMENT OF INFORMATICS
s
2
1
h
h
2 2
w =h*
1
w =h*1
y
D. Gesbert: IN357 Statistical Signal Processing15 of 17
Weights
phases
estimation
Application 3: Multi-antenna diversity
combining
UNIVERSITY
OF OSLO
X(n) = Hs(n) + V (n)
DEPARTMENT OF INFORMATICS
(6)
D. Gesbert: IN357 Statistical Signal Processing16 of 17
E|WoT X(n) − s(n)|2 minimum
Problem: Find Wo such that
where X(N ) collects the signals received on p antennas. s(n) is the
transmitted symbol sequence (e.g. s(n) = +/ − 1). His the channel
vector of size p. V (n) is the white noise vector.
Received signal model:
Antenna diversity combining (II)
UNIVERSITY
OF OSLO
DEPARTMENT OF INFORMATICS
D. Gesbert: IN357 Statistical Signal Processing17 of 17
NOTE: This receiver is equivalent to the minimum mean square error
(MMSE) receiver.
where σv2 is the variance of noise samples.
Wo = (H ∗H T + σv2I)−1H ∗
RxWo = rdx
(H ∗H T + σv2I)Wo = H ∗
Answer (Wiener-Hopf ):
Antenna diversity combining (III)
UNIVERSITY
OF OSLO