Visual and Auditory Systems
Tutorial 11
The Technion - Israel Institute of Technology
Electrical engineering faculty
Tutorial 11 – Signals Representation
Questions:
1D signals are represented using the following functions  x 
:
1
 f mn
 x   g  x  mD  cos  nWx  , 0  n   ;    m  

 2

 f mn  x   g  x  mD  cos  nWx  0  , 1  n   ;    m  
For integers m,n.
g  x  is a window function, as seen in figure 1:
WD  2
; 0 

3
2
The representation coefficients a1mn , amn
are complex numbers.
1
2
a. Show that the group  f mn
, f mn
 , which includes all the functions f mn1 , f mn2 is not
orthogonal. Find its biorthogonal group.
Directions: Assume the biorthogonal functions are also of the form:
 mn  h  x  mD  cos  nWx    .
1
2
b. Represent the following signals using the functions  f mn
, f mn
:
1  x   cos Wx 

1
2  x   
0
3
D
2
else
x
1
2
, f mn
c. Find an approximation for the following signal using the functions  f mn
 , for
n  0,1 :

cos Wx 
3  x   
0
0 x
D
2
else
d. Compare the representation error of 3  x  represented by f 011 alone, to the
representation error of 3  x  represented by f 011 and f 021 . Which representation is
better?
The approximation error between signals u and v is defined by:

E
 u  x  v  x
2
dx

1
Visual and Auditory Systems
Tutorial 11
The Technion - Israel Institute of Technology
Electrical engineering faculty
Solutions
a. Cosines in different frequencies are orthogonal, as are non-overlapping shifted
windows.
The only problem "spoiling" the orthogonality is  0 .
f
1
mn
 x, f  x
2
mn


 g  x  mD  cos  n x  cos  n x    dx 
2
0

 m 12  D

 m 12 
1
1
cos  2n x  0   cos 0   dx  D cos 0 
2
2
D
Therefore the group is not orthogonal:
1
D
1
f mn
, f mn2  D cos 0    0
2
4
Finding the bi-orthogonal group:
According to the directions, we shall choose a group of the form:
i
 mn
 x   Ki g  x  mD  cos  nWx  i 
In order to receive biorthogonaloity the following requirement must be fulfilled:
i
f mn
, klj   ij mk  nl
For m  k , there won't be an overlap between the windows g ; therefore the inner
product must be zero:
i
m  k  f mn
, knj  0
For n  1 :
1
1
f mn
, ml


 g  x  mD  cos  n x  cos  l x    dx 
2
1

 m 12 D

 m 12 
1
cos   n  l   x  1   cos   n  l   x  1   dx  0
2
D
f mn2 , ml2 

 g  x  mD  cos  n x    cos  l x    dx 
2
0
2

 m 12 D

 m 12 
1
cos   n  l   x  0   2   cos   n  l   x  0   2   dx  0
2
D
2
Visual and Auditory Systems
Tutorial 11
The Technion - Israel Institute of Technology
Electrical engineering faculty
For i  j :
f ,
1
mn

2
mn

 g  x  mD  cos  n x  cos  n x    dx  60
2
2

 m 12 D

 m 12 
1
D
cos  2n x   2   cos  2   dx  cos  2 
2
2
D
1
f mn2 , mn


 g  x  mD  cos  n x    cos  n x    dx 
2
0
1

 m 12 D

 m 12 
1
D
cos  2n x  0  1   cos  0  1   dx  cos 1  0 
2
2
D
In order for the biorthogonaloity requirement to be fulfilled, we shall demand for
both integrals to be equal to zero, so the result is:
f ,
1
mn
1
mn
 1  K1
 m 12  D

 m 12 
1 



cos  2n x  0    cos   0   dx 

2 
2
2

D



n0
 K1 D cos  2  0 




 1 K D cos     
n0
0

 2 1
2

 2
n0
 3D
 K1  
 4
n0
 3D
4
2
f mn2 , mn
1

K2 
3D
b.
1  x   cos  x  
a  1

 a1mn  0
 2
amn  0
1
m1

 g  x  mD  cos  x    f  x 
1
m1
m 
mn
m
n  1
m, n
 2  x  can be separated to a sum of 3 windows:

1
2  x   

0
3
D
1
1
1
2  g  x   g  x  D   g  x  D   f 00  x   f10  x   f 10  x 
else
x
1
1
a00
 a10
 a110  1

 a1mn  0
m, n  00,10, 10
 2
m, n
amn  0
3
Visual and Auditory Systems
Tutorial 11
The Technion - Israel Institute of Technology
Electrical engineering faculty
c.

cos Wx 
3  x   
0
0 x
D
2
else
For m  0 all the other windows do not overlap, therefore a1mn  0 .
D
a  3  x  ,
1
00
1
00
2

3D
 g  x  cos  x  cos  2  
4
3D
 g  x  cos  x  cos  x  
a  3  x  ,
2
01
4

3D
2
0

0
D
2
01
0
0
D
1
1
a01
 3  x  , 01

 dx  0
2
2
 dx  12
 g  x  cos  x  cos  x   2  dx  0
2
0
1 1
f 01
2
1
d. Calculating a02
:
 3  x  
D
a  3  x  ,
1
02
2
02
4

3D
 g  x  cos  x  cos  2 x  
2
0
1

In section c we received a01
E f
D
1
01
 
2
D
2
2

2
 dx  0
1
1
1
, therefore: 3  x   f 011  f 021 .
2
2
4
D
0
1
1
3  x   f 011 dx   cos 2  x  dx 
2
4
D
2
 cos  x  cos  2 x  
E f , f    

 dx 
2
4
D



2
2
0
1
01
0
1
02
D
2

0
2
1
D
 4 cos  x  dx  8
2
0
 cos  x  cos  2 x  


 dx 
2
4


2
D
1 1 D 5
D
   
8
 4 16  2 32
Meaning the representation error is bigger in the second representation!
Note that contrary to a representation using an orthogonal basis, where adding
members necessarily reduces the error, in a representation using a
biorthogonal basis adding members not necessarily reduces the error!
4
Visual and Auditory Systems
Tutorial 11
The Technion - Israel Institute of Technology
Electrical engineering faculty
Appendix A – Biorthogonal Spaces
Inner product
V is a vector space.
A function on 2 vectors V  u, v shall be called an inner product if the following
requirements are fulfilled:
1. u1  u2 , v  u1 , v  u2 , v
2.
cu, v  c u, v (c is const )
3.
u , v  v, u
4.
u, u  0
5.
u, u  0  u  0
*
Norm
An operation that fulfills the next requirements is a norm:
1. u  0
u 0u 0
2.
cu  c  u
3.
uv  u  v
Metric - distance function
d  u, v  is called a metric if the following requirements are fulfilled:
1. d  u, v   d  v, u 
2. d  u, v   d  u, s   d  s, v 
3. d  u, v   0
4. d  u, v   0  u  v
Sequences
 fn  is a sequence, V  f n .
The sequence  f n  converges to the limit V  f if and only if for every   0 there is
an N, so that for every n  N :
d  fn , f   
lim d  f n , f   0
n 
Cauchy sequence
A sequence  f n  is called a Cauchy sequence if for every   0 there is an N, so that
for every m, n  N d  f n , f    .
Every convergent sequence is a Cauchy sequence, but not vice versa.
5
Visual and Auditory Systems
Tutorial 11
The Technion - Israel Institute of Technology
Electrical engineering faculty
Hilbert space
An inner product space in which every Cauchy sequence is convergent.
Orthogonality
V is an inner product space. V  x, y .
x,y are orthogonal if and only if x, y  0 .
A sequence n  is orthogonal if for every m  n , m , n  0 .
If also n , n  1 the sequence is called orthonormal .
Complete sequence
A sequence n  is complete if only the member 0 is orthogonal to all the other
members.
Basis
V is an n-dimensional vector space.
If there are linearly independent vectors e1 , e2 ,..., en that span the whole space V, they
are said to be a basis for V.
n
For every v V we can write: v   ai ei .
i 1
Theorem
1. A complete orthogonal sequence is a basis in its space.
2. In every Hilbert space there is a complete orthogonal sequence.
Representation of a member f by an orthonormal sequence
A sequence n  is a complete orthonormal sequence.
Every member f can be represented by: f   ann .
n
The representation coefficients are: an  f , n .
Biorthonormality
Two sequences n  ,  n  are called biorthonormal if and only if n , n   mn .
Theorem
If n  is a basis to H (a Hilbert space), it has a single biorthonormal sequence  n 
and it is a basis.
Conclusion: Every f  H can be represented using the representation coefficients an
of the basis n  , according to f   ann , where an   n , f .
n
6
Visual and Auditory Systems
Tutorial 11
The Technion - Israel Institute of Technology
Electrical engineering faculty
Appendix B – The Uncertainty Principle

f  x  is a function that satisfies
f  x  dx   .

2


The Fourier transform is F    f  x  e j x dx .


 x f  x
Consider

dx  0 (meaning the function is centered on zero)
2
We shall define the effective width of the function in the space and frequency domain
as its second normalized momentum in each of the following axis:

 x 
2

x
f  x  dx
2
2


 f  x
2
dx

2
1
  x 2 f  x  dx
E 


  
2



2
F   d
2

 F  

2
d

1

2 E 
2
F   d
2


Prove:
 0 the expression x   
For every f  x  which satisfies x f  x  
x 
1
is true.
2
Proof:
For the sake of simplicity we shall assume that f  x  is real and normalized, meaning

E
 f  x
2
dx  1 .

Cauchy–Schwarz inequality: u, v  v  u .
In the L2  a, b space:
2

 u  x  v  x  dx

Substitution: u  x   xf  x  , v  x  


 u  x

df  x 
.
dx
2

dx   v  x  dx
2



df  x 
 df  x  
dx   x 2 f 2  x  dx   
Therefore:    xf  x 
 dx
dx
dx 


 
Starting with the left section:



df  x 
1 2
1 2
1
1
xf
x
dx

xf
x

E

   dx integration by parts 2     2 f  x  dx  lim x0f x 0  2 E 1 normalized
2
 

2
2
7
Visual and Auditory Systems
Tutorial 11
The Technion - Israel Institute of Technology
Electrical engineering faculty
Right section:
 df 
Using the fact that F    j F   , and according to Parseval's theorem:
 dx 



df  x 
1
dx 
dx
2
2



 df  x  
1
F
 d 
2
 dx 
2


2
F   d    
2
2

Placing these results in the inequality   :
2
2
2
1
    x     
2
1
 x   
2
This inequality is also called the "uncertainty principle" because of the analogy to
the Heisenberg uncertainty principle from quantum mechanics.
8