Chapter 5: Signal Space Analysis
CHAPTER 5
SIGNAL SPACE ANALYSIS
Digital Communication Systems 2012
R.Sokullu
1/26
Chapter 5: Signal Space Analysis
Outline
• 5.1 Introduction
• 5.2 Geometric Representation of Signals
– Gram-Schmidt Orthogonalization Procedure
• 5.3 Conversion of the AWGN into a Vector Channel
• 5.4 Likelihood Functions
• 5.5 Maximum Likelihood Decoding
• 5.6 Correlation Receiver
• 5.7 Probability of Error
Digital Communication Systems 2012
R.Sokullu
2/26
Chapter 5: Signal Space Analysis
Likelihood Functions
• In a sense, likelihood works backwards from probability: given parameter
B, we use the conditional probability P(A|B) to reason about outcome A,
and given outcome A, we use the likelihood function L(B|A) to reason about
parameter B. This mode of reasoning is formalized in Bayes' theorem:
• A likelihood function is a conditional probability function considered as a
function of its second argument with its first argument held fixed, thus:
and also any other function proportional to such a function. That is, the likelihood
function for B is the equivalence class of functions
Digital Communication Systems 2012
R.Sokullu
3/26
Chapter 5: Signal Space Analysis
Likelihood Functions
• As we discussed in the previous class, the
conditional probability density functions
fX(x|mi), I = 1, 2, 3, …M are the very
characterization of the AWGN channel.
• They express the functional dependence of the
observation vector x on the transmitted
message symbol mi. (known as the transmitted
message symbol)
Digital Communication Systems 2012
R.Sokullu
4/26
Chapter 5: Signal Space Analysis
However,
• If we have the observation vector given, and we
want to define the transmitted message signal, then
we have the reverse situation
• We introduce the “likelihood function” L(mi) as:
L
(
m
)

f
(
x
/
m
)
, i

1
,
2
,
.
.
.
.
.
,
M
(
5
.
4
9
)
i
X
i
Looks very
similar????
Yes, but meaning is
different…
• Or log likelihood function ..l(mi) as:
l
(
m
)

l
o
g
L
(
m
)
, i

1
,
2
,
.
.
.
.
.
,
M
(
5
.
5
0
)
i
i
Digital Communication Systems 2012
R.Sokullu
5/26
Chapter 5: Signal Space Analysis
Log-Likelihood Function of AWGN
Channel
• Substitute 5.46 into 5.50:

Vector presentation of
the AWGN channel
N


1

N
/
2
2
f
(
x
/
m
)

(
N
)
e
x
p()

x
,


x
i
0
js
i
j
i
=
1
,
2
,
.
.
.
.
,
M
(
5
.
4
6
)
N
j

1
0


l
(
m
)

l
o
g
L
(
m
)
, i

1
,
2
,
.
.
.
.
.
,
M
(
5
.
5
0
)
i
i
• where sij, j = 1, 2, 3, ..N are the elements of the signal
vector si, representing the message symbol mi.
Digital Communication Systems 2012
R.Sokullu
6/26
Chapter 5: Signal Space Analysis
So,
N
1
2
l
(
m
)


(
x

s
)

1
,
2
,
.
.
.
.
.
,
M
(
5
.
5
1
)

i
j i
j,i
N
j

1
0
which is the log likelihood function of the AWGN channel..
Digital Communication Systems 2012
R.Sokullu
7/26
Chapter 5: Signal Space Analysis
Outline
• 5.1 Introduction
• 5.2 Geometric Representation of Signals
– Gram-Schmidt Orthogonalization Procedure
• 5.3 Conversion of the AWGN into a Vector Channel
• 5.4 Likelihood Functions
• 5.5 Maximum Likelihood Decoding
• 5.6 Correlation Receiver
• 5.7 Probability of Error
Digital Communication Systems 2012
R.Sokullu
8/26
Chapter 5: Signal Space Analysis
5.5 Maximum Likelihood Decoding
• Defining the problem
– Suppose that in each time slot duration of T seconds, one of
M possible signals, s1(t), s2(t), …sM(t) is transmitted with
equal probability, 1/M.
– As described in the previous part, for the vector
representation, the signal si(t), i=1, 2, …M is applied to a
bank of correlators, with a common input and supplied with
a suitable set of N orthogonal basis functions, N. The
resulting output defines the signal vector si.
– We represent each signal si(t) as a point in the Euclidian
space, N ≤ M (referred to as transmitted signal point or
message point).The set of message points corresponding to
the set of transmitted signals si(t) {i = 1 to M} is called
signal constellation.
Digital Communication Systems 2012
R.Sokullu
9/26
Chapter 5: Signal Space Analysis
Figure 5.3
(a) Synthesizer for generating the signal si(t). (b) Analyzer for
generating the set of signal vectors si.
Digital Communication Systems 2012
R.Sokullu
10/26
Chapter 5: Signal Space Analysis
– The received signal x(t) is applied to a bank of N
correlators (Fig. 5.3b) and the correlator outputs
define the observation vector x.
– On the receiving side the representation of the
received signal x(t) is complicated by the additive
noise w(t).
– As we discussed the previous class, the vector x
differs from the vector si by the noise vector w.
– However only the portion of it which interferes
with the detection process is of importance to us,
and this is fully described by w(t).
Digital Communication Systems 2012
R.Sokullu
11/26
Chapter 5: Signal Space Analysis
• Based on the observation vector x we may represent
the received signal signal x(t) by a point in the same
Euclidian space used to represent the transmitted
signal.
Digital Communication Systems 2012
R.Sokullu
12/26
Chapter 5: Signal Space Analysis
• For a given observation vector x we have to make a
decision m' = mi
• The decision is based on the criterion to minimize
the probability of error in mapping each observation
vector into a decision.
• So the optimum decision rule is:
ˆ
S
e
t
m
=
m
f
ii
P
(
m
s
e
n
t
/
x
)(

P
m
s
e
n
t
/
x
)
f
o
r
a
l
l
k

i(
5
.
5
3
)
i
k
Digital Communication Systems 2012
R.Sokullu
13/26
Chapter 5: Signal Space Analysis
• The same rule can be more explicitly expressed
using the a priori probabilities of the transmitted
signals as:
ˆ
S
e
t
m
=
m
f
ii
Conditional pdf of
observation vector X
given mk was transmitted
p
f
(
x
/
m
)
k
X
k
i
s
m
a
x
i
m
u
m
f
o
r
k
=
i(
5
.
5
4
)
f
(
x
)
X
a priori probability
of transmitting mk
Unconditional pdf
of observation
vector X
Digital Communication Systems 2012
R.Sokullu
14/26
Chapter 5: Signal Space Analysis
• Thus we can conclude, according to the definition of
likelihood functions, the likelihood function l(mk) will be
maximum for k = i.
• So the decision rule using the likelihood function will be
formulated as:
ˆ
S
e
t
m
=
m
f
ii
l
()
m
s
m
a
x
i
m
u
m
f
o
r
k
=
i(
5
.
5
5
)
ki
• For a graphical representation of the maximum likelihood rule
we introduce the following:
– Observation space – Z, N-dimensional, consisting of all possible
observation vectors x
– Z is partitioned into M decision regions, Z1, Z2, .. ZM
O
b
s
e
r
v
a
t
i
o
n
v
e
c
t
o
r
x
l
i
e
s
r
e
g
i
o
n
Z
f
ii
l
()
m
s
m
a
x
i
m
u
m
f
o
r
k
=
i (
5
.
5
6
)
ki
Digital Communication Systems 2012
R.Sokullu
15/26
Chapter 5: Signal Space Analysis
For the AWGN channel..
• Based on the log-likelihood
function, of the AWGN channel,
l(mk) will be max when the term:
is minimized by k = i.
• Decision rule for
AWGN:
N
2
(
x

s
),
j ij
j
1
O
b
s
e
r
v
a
t
i
o
n
v
e
c
t
o
r
x
l
i
e
s
r
e
g
i
o
n
Z
f
ii
N
2
(
xs

)
s
m
i
n
i
m
u
m
f
o
r
k
=
i (
5
.
5
7
)

j
k
j ,i
j

1
• Or using Euclidian
space notation
O
b
s
e
r
v
a
t
i
o
n
v
e
c
t
o
r
x
l
i
e
s
r
e
g
i
o
n
Z
f
ii
t
h
e
E
u
c
l
i
d
e
a
n
d
i
s
t
a
n
c
exs

k
i
s
m
i
n
i
m
u
m
f
o
r
k
=
i
Digital Communication Systems 2012
R.Sokullu
(
5
.
5
9
)
16/26
Chapter 5: Signal Space Analysis
Finally,
• (5.59) states that the maximum likelihood decision rule is
simply to choose the message point closest to the received
signal point.
• After few re-organizations we get: (left as homework brain
gymnastic exercise for you)
O
b
s
e
r
v
a
t
i
o
n
v
e
c
t
o
r
x
l
i
e
sr
i
n
e
g
i
o
n
Z
f
ii
N
1
x
s
E
i
s
m
a
x
i
m
u
m
f
o
r
k
=
i (
5
.
6
1
)

jk
j
k
2
j

1
N
2
E

s
k 
k
j
(
5
.6
2
)
j
1
Energy of the
transmitted signal
sk(t)
Digital Communication Systems 2012
R.Sokullu
17/26
Chapter 5: Signal Space Analysis
Figure 5.8
Illustrating the
partitioning of the
observation space
into decision regions
for the case when
N  2 and M  4;
it is assumed that the
M transmitted
symbols are equally
likely.
Digital Communication Systems 2012
R.Sokullu
18/26
Chapter 5: Signal Space Analysis
Outline
• 5.1 Introduction
• 5.2 Geometric Representation of Signals
– Gram-Schmidt Orthogonalization Procedure
• 5.3 Conversion of the AWGN into a Vector Channel
• 5.4 Likelihood Functions
• 5.5 Maximum Likelihood Decoding
• 5.6 Correlation Receiver
• 5.7 Probability of Error
Digital Communication Systems 2012
R.Sokullu
19/26
Chapter 5: Signal Space Analysis
5.6 Correlation Receiver
• Based on the theoretical assumptions made in the previous
class we define the correlator at the receiver side.
• It can be implemented as a optimum receiver that consists of
two parts:
– Detector part – M product-integrators supplied with the corresponding
set of coherent reference signals (orthogonal basis functions),
generated locally. It operates on the received signal s(t) to produce the
observation vector x for 0≤ t ≤ T.
– Receiver part – signal transmission decoder – which is implemented in
the form of a maximum likelihood decoder, operating on the
observation vector x to produce the estimate m‘ of the transmitted
symbol mi in a way to minimize the average probability of symbol
error. According to (5.61) the N elements of the observation vector x
are multiplied by the N elements of each of the M signal vectors s1, s2,
..sM and then summed up to produce the inner products [xTsk|k=1,2..M].
Largest of the resulting numbers is selected.
Digital Communication Systems 2012
R.Sokullu
20/26
Chapter 5: Signal Space Analysis
Figure 5.9
(a) Detector or
demodulator.
(b) Signal transmission
decoder.
Digital Communication Systems 2012
R.Sokullu
21/26
Chapter 5: Signal Space Analysis
Note:
• The detector shown
in Fig. 5.9a is based
on correlators.
• Alternatively,
matched filters,
discussed in Chap.
4.2 may be used to
produce the required
observation vector x.
Detector part of matched filter receiver;
the signal transmission decoder is as shown in Fig. 5.9b
Digital Communication Systems 2012
R.Sokullu
22/26
Chapter 5: Signal Space Analysis
Outline
• 5.1 Introduction
• 5.2 Geometric Representation of Signals
– Gram-Schmidt Orthogonalization Procedure
• 5.3 Conversion of the AWGN into a Vector Channel
• 5.4 Likelihood Functions
• 5.5 Maximum Likelihood Decoding
• 5.6 Correlation Receiver
• 5.7 Probability of Error
Digital Communication Systems 2012
R.Sokullu
23/26
Chapter 5: Signal Space Analysis
5.7 Probability of Error
• To complete the statistical characterization of the
correlation receiver (Fig. 5.9) we need to discuss its
noise performance.
• Using the assumptions made before, we can define the
average probability of error Pe as:
M
P
p
(xd
o
e
sn
o
tlie
in
Z
iv
e
n
m
,
e=
iP
i |g
i)
j
1
1M
= 
P
(xd
o
e
sn
o
tlie
in
Z
iv
e
n
m
i |g
i)
M
i
1
(5
.6
7
)
1M

1 
P
(xlie
sin
Z
iv
e
n
m
i |g
i)
M
i
1
Digital Communication Systems 2012
R.Sokullu
24/26
Chapter 5: Signal Space Analysis
• Using the likelihood function this can be re-written
as:
M
1
P
1

fx
(|
m
)
d
x
(
5
.
6
8
)

e
x
i

M
i

1
Z
i
• The probability of error is invariant to rotation and
translation of the signal constellation.
– In maximum likelihood detection the probability of
symbol error Pe depends solely on the Euclidian distances
between the message points in the constellation
– The additive Gaussian noise is spherically symmetric in
all directions in the signal space.
Digital Communication Systems 2012
R.Sokullu
25/26
Chapter 5: Signal Space Analysis
Conclusions:
• This chapter presents a systematic procedure for the analysis
of signals in a vector space.
• The basic idea of the approach is to represent each member of
a set of transmitted signals by an N-dimensional vector, where
N is the number of orthogonal basis functions, needed for the
unique representation of the transmitted signals.
• The set of signal vectors defines the signal constellation, the
N-dimensional space defines the signal space.
• It is the theoretical basis for the design of a digital
communication receiver in the presence of AWGN. The
procedure is based on the theory of maximum likelihood
detection.
• The average probability of symbol error is defined as Pe. It is
dominated by the nearest neighbors to the transmitted signal.
Digital Communication Systems 2012
R.Sokullu
26/26