Chapter 5: Signal Space Analysis
CHAPTER 5
SIGNAL SPACE ANALYSIS
Digital Communication Systems 2012
R.Sokullu
1/45
Chapter 5: Signal Space Analysis
Outline
• 5.1 Introduction
• 5.2 Geometric Representation of Signals
– Gram-Schmidt Orthogonalization Procedure
• 5.3 Conversion of the AWGN into a Vector Channel
• 5.4 Maximum Likelihood Decoding
• 5.5 Correlation Receiver
• 5.6 Probability of Error
Digital Communication Systems 2012 R.Sokullu
2/45
Chapter 5: Signal Space Analysis
Introduction – the Model
• We consider the following model of a generic
transmission system (digital source):
– A message source transmits 1 symbol every T sec
– Symbols belong to an alphabet M (m1, m2, …mM)
• Binary – symbols are 0s and 1s
• Quaternary PCM – symbols are 00, 01, 10, 11
Digital Communication Systems 2012 R.Sokullu
3/45
Chapter 5: Signal Space Analysis
Transmitter Side
• Symbol generation (message) is probabilistic, with
a priori probabilities p1, p2, .. pM. or
• Symbols are equally likely
• So, probability that symbol mi will be emitted:
i  P(mi )
=
1
for i=1,2,....,M
M
(5.1)
Digital Communication Systems 2012 R.Sokullu
4/45
Chapter 5: Signal Space Analysis
• Transmitter takes the symbol (data) mi (digital
message source output) and encodes it into a
distinct signal si(t).
• The signal si(t) occupies the whole slot T allotted
to symbol mi.
• si(t) is a real valued energy signal (???)
T
Ei   si2 (t )dt , i=1,2,....,M
(5.2)
0
Digital Communication Systems 2012 R.Sokullu
5/45
Chapter 5: Signal Space Analysis
• Transmitter takes the symbol (data) mi (digital
message source output) and encodes it into a
distinct signal si(t).
• The signal si(t) occupies the whole slot T allotted
to symbol mi.
• si(t) is a real valued energy signal (signal with
finite energy)
T
Ei   si2 (t )dt , i=1,2,....,M
(5.2)
0
Digital Communication Systems 2012 R.Sokullu
6/45
Chapter 5: Signal Space Analysis
Channel Assumptions:
• Linear, wide enough to accommodate the signal si(t)
with no or negligible distortion
• Channel noise is w(t) is a zero-mean white Gaussian
noise process – AWGN
– additive noise
– received signal may be expressed as:
x(t )  si (t )  w(t ),
0  t  T 


i=1,2,....,M 
Digital Communication Systems 2012 R.Sokullu
(5.3)
7/45
Chapter 5: Signal Space Analysis
Receiver Side
• Observes the received signal x(t) for a duration of time T sec
• Makes an estimate of the transmitted signal si(t) (eq. symbol mi).
• Process is statistical
– presence of noise
– errors
• So, receiver has to be designed for minimizing the average
probability of error (Pe)
What is this?
M
Pe =
 p P(mˆ  m
i 1
cond. error probability
given ith symbol was
sent
i
i
/ mi )
(5.4)
Symbol sent
Digital Communication Systems 2012 R.Sokullu
8/45
Chapter 5: Signal Space Analysis
Outline
• 5.1 Introduction
• 5.2 Geometric Representation of Signals
– Gram-Schmidt Orthogonalization Procedure
• 5.3 Conversion of the AWGN into a Vector Channel
• 5.4 Maximum Likelihood Decoding
• 5.5 Correlation Receiver
• 5.6 Probability of Error
Digital Communication Systems 2012 R.Sokullu
9/45
Chapter 5: Signal Space Analysis
5.2. Geometric Representation of
Signals
• Objective: To represent any set of M energy signals
{si(t)} as linear combinations of N orthogonal basis
functions, where N ≤ M
• Real value energy signals s1(t), s2(t),..sM(t), each of
duration T sec
Orthogonal basis
function
N
si (t )   sij j (t ),
j 1
0  t  T



i==1,2,....,M 
(5.5)
coefficient
Energy signal
Digital Communication Systems 2012 R.Sokullu
10/45
Chapter 5: Signal Space Analysis
• Coefficients:
T
sij   si (t ) j (t )dt ,
0
i=1,2,....,M 

 (5.6)
 j=1,2,....,M 
• Real-valued basis functions:
1 if i  j 
0 i (t ) j (t )dt  ij  0 if i  j 
T
Digital Communication Systems 2012 R.Sokullu
(5.7)
11/45
Chapter 5: Signal Space Analysis
• The set of coefficients can be viewed as a Ndimensional vector, denoted by si
• Bears a one-to-one relationship with the
transmitted signal si(t)
Digital Communication Systems 2012 R.Sokullu
12/45
Chapter 5: Signal Space Analysis
Figure 5.3
(a) Synthesizer for generating the signal si(t). (b) Analyzer
for generating the set of signal vectors si.
Digital Communication Systems 2012 R.Sokullu
13/45
Chapter 5: Signal Space Analysis
So,
• Each signal in the set si(t) is completely
determined by the vector of its coefficients
 si1 
s 
 i2 
. 
si    ,
. 
. 
 
 siN 
i  1,2,....,M
Digital Communication Systems 2012 R.Sokullu
(5.8)
14/45
Chapter 5: Signal Space Analysis
Finally,
• The signal vector si concept can be extended to 2D, 3D etc. Ndimensional Euclidian space
• Provides mathematical basis for the geometric representation
of energy signals that is used in noise analysis
• Allows definition of
– Length of vectors (absolute value)
– Angles between vectors
– Squared value (inner product of si with itself)
si
2
 siT si
N
=  sij2 ,
Matrix
Transposition
i  1,2,....,M
(5.9)
j 1
Digital Communication Systems 2012 R.Sokullu
15/45
Chapter 5: Signal Space Analysis
Figure 5.4
Illustrating the geometric
representation of signals
for the case when N  2
and M  3.
(two dimensional space,
three signals)
Digital Communication Systems 2012 R.Sokullu
16/45
Chapter 5: Signal Space Analysis
Also,
What is the relation between the vector representation
of a signal and its energy value?
• …start with the definition
of average energy in a
signal…(5.10)
T
Ei   si2 (t )dt
(5.10)
0
N
• Where si(t) is as in (5.5):
si (t )   sij j (t ), (5.5)
j 1
Digital Communication Systems 2012 R.Sokullu
17/45
Chapter 5: Signal Space Analysis
N
 N

Ei    sij j (t )   sikk (t )  dt

  k 1
0  j 1
T
• After substitution:
N
• After regrouping:
Ei  
• Φj(t) is orthogonal, so
finally we have:
j 1
T
N
 s s   (t ) (t )dt
k 1
ij ik
j
(5.11)
0
N
Ei   s
j 1
k
2
ij
= si
2
(5.12)
The energy of a signal
is equal to the squared
length of its vector
Digital Communication Systems 2012 R.Sokullu
18/45
Chapter 5: Signal Space Analysis
Formulas for two signals
• Assume we have a pair of signals: si(t) and sj(t), each
represented by its vector,
• Then:
T
sij   si (t )sk (t )dt  s s
0
Inner product of the signals
is equal to the inner product
of their vector
representations [0,T]
T
i k
(5.13)
Inner product is invariant
to the selection of basis
functions
Digital Communication Systems 2012 R.Sokullu
19/45
Chapter 5: Signal Space Analysis
Euclidian Distance
• The Euclidean distance between two points
represented by vectors (signal vectors) is equal to
||si-sk|| and the squared value is given by:
N
si  s k = (sij -skj ) 2
2
(5.14)
j 1
T
=  ( si (t )  sk (t )) 2 dt
0
Digital Communication Systems 2012 R.Sokullu
20/45
Chapter 5: Signal Space Analysis
Angle between two signals
• The cosine of the angle Θik between two signal vectors si and
sk is equal to the inner product of these two vectors, divided
by the product of their norms:
T
i k
s s
cosik 
si sk
(5.15)
• So the two signal vectors are orthogonal if their inner product
siTsk is zero (cos Θik = 0)
Digital Communication Systems 2012 R.Sokullu
21/45
Chapter 5: Signal Space Analysis
Schwartz Inequality
• Defined as:



 
2
s1 (t )s2 (t )dt


2
 1
s (t )dt
 


s22 (t )dt
 (5.16)
• accept without proof…
Digital Communication Systems 2012 R.Sokullu
22/45
Chapter 5: Signal Space Analysis
Outline
• 5.1 Introduction
• 5.2 Geometric Representation of Signals
– Gram-Schmidt Orthogonalization Procedure
• 5.3 Conversion of the AWGN into a Vector Channel
• 5.4 Maximum Likelihood Decoding
• 5.5 Correlation Receiver
• 5.6 Probability of Error
Digital Communication Systems 2012 R.Sokullu
23/45
Chapter 5: Signal Space Analysis
Gram-Schmidt Orthogonalization
Procedure
Assume a set of M energy signals denoted by s1(t), s2(t), .. , sM(t).
1. Define the first basis function
starting with s1 as: (where E is the
energy of the signal) (based on
5.12)
2. Then express s1(t) using the basis
function and an energy related
coefficient s11 as:
3. Later using s2 define the
coefficient s21 as:
s1 (t )
1 (t ) 
E1
(5.19)
s1 (t )  E11 (t ) =s111 (t ) (5.20)
T
s21   s2 (t )1 (t )dt
0
Digital Communication Systems 2012 R.Sokullu
(5.21)
24/45
Chapter 5: Signal Space Analysis
4.
5.
If we introduce the
intermediate function g2 as:
We can define the second
basis function φ2(t) as:
g2 (t )  s2 (t )  s211 (t ) (5.22)
Orthogonal to φ1(t)
2 (t ) 
g2 (t )

T
0
6.
•
Which after substitution of
g2(t) using s1(t) and s2(t) it
becomes:
Note that φ1(t) and φ2(t) are
orthogonal that means:
2 (t ) 

T
0

T
0
(5.23)
g22 (t )dt
s2 (t )  s211 (t )
E2  s
2
21
22 (t )dt  1
(5.24)
(Look at 5.23)
1 (t )2 (t )dt  0
Digital Communication Systems 2012 R.Sokullu
25/45
Chapter 5: Signal Space Analysis
And so on for N dimensional space…,
• In general a basis function can be defined using the
following formula:
i 1
gi (t )  si (t )   sij - j (t)
(5.25)
j 1
• where the coefficients can be defined using:
T
sij   si (t ) j (t )dt ,
0
j  1, 2,....., i  1
Digital Communication Systems 2012 R.Sokullu
(5.26)
26/45
Chapter 5: Signal Space Analysis
Special case:
• For the special case of i = 1 gi(t) reduces to si(t).
General case:
• Given a function gi(t) we can define a set of basis
functions, which form an orthogonal set, as:
i (t ) 
gi (t )

T
0
,
i  1, 2,....., N
(5.27)
gi2 (t )dt
Digital Communication Systems 2012 R.Sokullu
27/45
Chapter 5: Signal Space Analysis
Outline
• 5.1 Introduction
• 5.2 Geometric Representation of Signals
– Gram-Schmidt Orthogonalization Procedure
• 5.3 Conversion of the AWGN into a Vector Channel
• 5.4 Maximum Likelihood Decoding
• 5.5 Correlation Receiver
• 5.6 Probability of Error
Digital Communication Systems 2012 R.Sokullu
28/45
Chapter 5: Signal Space Analysis
Conversion of the Continuous AWGN
Channel into a Vector Channel
• Suppose that the si(t) is
not any signal, but
specifically the signal at
the receiver side, defined
in accordance with an
AWGN channel:
• So the output of the
correlator (Fig. 5.3b) can
be defined as:
x (t )  si (t )  w(t ),
0  t  T 


i=1,2,....,M 
(5.28)
T
x i   x (t ) j (t ) dt
0
=sij  wi ,
j  1, 2,....., N (5.29)
Digital Communication Systems 2012 R.Sokullu
29/45
Chapter 5: Signal Space Analysis
T
sij   si (t )i (t )dt (5.30)
0
deterministic quantity
contributed by the
transmitted signal si(t)
T
wi   w(t )i (t )dt
(5.31)
0
random quantity
sample value of the
variable Wi due to noise
Digital Communication Systems 2012 R.Sokullu
30/45
Chapter 5: Signal Space Analysis
Now,
• Consider a random
N
process X1(t), with x1(t), a
(5.32)
sample function which is x(t )  x(t )   x ji (t )
j 1
related to the received
N
signal x(t) as follows:
x(t )  x(t )   ( sij  w j ) j (t )
j 1
• Using 5.28, 5.29 and 5.30
N
and the expansion 5.5 we
=w(t )   w j j (t )
get:
j 1
=w(t )
(5.33)
which means that the sample function x1(t) depends only on
the channel noise!
Digital Communication Systems 2012 R.Sokullu
31/45
Chapter 5: Signal Space Analysis
• The received signal can
be expressed as:
N
x(t )   x ji (t )  x(t )
j 1
N
  x ji (t )  w(t )
(5.34)
j 1
NOTE: This is an expansion similar to the one
in 5.5 but it is random, due to the additive
noise.
Digital Communication Systems 2012 R.Sokullu
32/45
Chapter 5: Signal Space Analysis
Statistical Characterization
• The received signal (output of the correlator of
Fig.5.3b) is a random signal. To describe it we need
to use statistical methods – mean and variance.
• The assumptions are:
– X(t) denotes a random process, a sample function of which
is represented by the received signal x(t).
– Xj(t) denotes a random variable whose sample value is
represented by the correlator output xj(t), j = 1, 2, …N.
– We have assumed AWGN, so the noise is Gaussian, so X(t)
is a Gaussian process and being a Gaussian RV, X j is
described fully by its mean value and variance.
Digital Communication Systems 2012 R.Sokullu
33/45
Chapter 5: Signal Space Analysis
Mean Value
• Let Wj, denote a random variable, represented by its
sample value wj, produced by the jth correlator in
response to the Gaussian noise component w(t).
• So it has zero mean (by definition of the AWGN
model)
 x  E  X j 
j
• …then the mean of
Xj depends only on
sij:
=E  sij  W j 
=sij  E[W j ]
 x = sij
j
Digital Communication Systems 2012 R.Sokullu
(5.35)
34/45
Chapter 5: Signal Space Analysis
Variance
• Starting from the definition,
we substitute using 5.29 and
5.31
 x2  var[ X j ]
i
=E ( X j  sij )2 
=E W j2 
T
wi   w(t )i (t )dt
(5.31)
0
T
 x2 = 
i
o
T
T
T


2
 xi =E   W (t ) j (t )dt  W (u ) j (u )du 
0
0

  (t ) (u) E[W (t )W (u)]dtdu
i
(5.36)
j
T
=E  
o

0  j (t )i (u)W (t )W (u)dtdu 
T
(5.37)
0
T
=E  
o

0  j (t )i (u) Rw (t , u)dtdu 
T
(5.38)
Autocorrelation function of
the noise process
Digital Communication Systems 2012 R.Sokullu
35/45
Chapter 5: Signal Space Analysis
• It can be expressed as:
(because the noise is
N0
R
(
t
,
u
)

 (t  u )
(5.39)
w
stationary and with a
2
constant power spectral
density)
T T
• After substitution for
the variance we get:
N0
 =
2
2
xi
   (t ) (u) (t  u)dtdu
i
o
j
0
T
N0 2
=
 j (t )dt

2 0
(5.40)
• And since φj(t) has unit
N0
2
for all j (5.41)
energy for the variance  x =
2
we finally have:
• Correlator outputs, denoted by Xj have variance
equal to the power spectral density N0/2 of the noise
process W(t).
i
Digital Communication Systems 2012 R.Sokullu
36/45
Chapter 5: Signal Space Analysis
Properties (without proof)
• Xj are mutually uncorrelated
• Xj are statistically independent (follows from above
because Xj are Gaussian)
• and for a memoryless channel the following equation
is true:
N
f x ( x / mi )   f x j ( x j / mi ),
i=1,2,....,M (5.44)
j 1
Digital Communication Systems 2012 R.Sokullu
37/45
Chapter 5: Signal Space Analysis
• Define (construct) a vector X of N random variables, X1, X2,
…XN, whose elements are independent Gaussian RV with
mean values sij, (output of the correlator, deterministic part of
the signal defined by the signal transmitted) and variance
equal to N0/2 (output of the correlator, random part,
calculated noise added by the channel).
• then the X1, X2, …XN , elements of X are statistically
independent.
• So, we can express the conditional probability of X, given si(t)
(correspondingly symbol mi) as a product of the conditional
density functions (fx) of its individual elements fxj.
NOTE: This is equal to finding an expression of the probability
of a received symbol given a specific symbol was sent,
assuming a memoryless channel
Digital Communication Systems 2012 R.Sokullu
38/45
Chapter 5: Signal Space Analysis
• …that is:
N
f x ( x / mi )   f x j ( x j / mi ),
i=1,2,....,M (5.44)
j 1
• where, the vector x and the scalar xj, are sample
values of the random vector X and the random
variable Xj.
Digital Communication Systems 2012 R.Sokullu
39/45
Chapter 5: Signal Space Analysis
N
f x ( x / mi )   f x j ( x j / mi ),
i=1,2,....,M (5.44)
j 1
Vector x and scalar xj
are sample values of
the random vector X
and the random
variable Xj
Vector x is called
observation vector
Scalar xj is called
observable element
Digital Communication Systems 2012 R.Sokullu
40/45
Chapter 5: Signal Space Analysis
• Since, each Xj is Gaussian with mean sj and variance
N0/2
f x j ( x / mi )  ( N0 )
N /2
 1
2
exp  ( x j  sij )  ,
 N0

j=1,2,....,N
i=1,2,....,M
(5.45)
• we can substitute in 5.44 to get 5.46:
f x ( x / mi )  ( N0 )
N / 2
 1 N
2
exp   ( x j  sij )  ,
 N0 j 1

Digital Communication Systems 2012 R.Sokullu
i=1,2,....,M (5.46)
41/45
Chapter 5: Signal Space Analysis
• If we go back to the formulation of the received
signal through a AWGN channel 5.34
N
x(t )   x ji (t )  x(t )
j 1
N
  x ji (t )  w(t )
(5.34)
j 1
The vector that we
have constructed fully
defines this part
Only projections of the noise onto
the basis functions of the signal set
{si(t)Mi=1 affect the significant
statistics of the detection problem
Digital Communication Systems 2012 R.Sokullu
42/45
Chapter 5: Signal Space Analysis
Finally,
• The AWGN channel, is equivalent to an Ndimensional vector channel, described by the
observation vector
x  si  w,
i  1, 2,....., M
Digital Communication Systems 2012 R.Sokullu
(5.48)
43/45
Chapter 5: Signal Space Analysis
Outline
• 5.1 Introduction
• 5.2 Geometric Representation of Signals
– Gram-Schmidt Orthogonalization Procedure
• 5.3 Conversion of the AWGN into a Vector Channel
• 5.4 Maximum Likelihood Decoding
• 5.5 Correlation Receiver
• 5.6 Probability of Error
Digital Communication Systems 2012 R.Sokullu
44/45
Chapter 5: Signal Space Analysis
Maximum Likelihood Decoding
• to be continued….
Digital Communication Systems 2012 R.Sokullu
45/45