Hossein Sameti
Department of Computer Engineering
Sharif University of Technology

Our focus in the next few lectures
Hossein Sameti, CE, SUT, Fall 1992 2

•
A discrete-time signal is a function of independent integer variables.
• x(n) is not defined at instants between two successive samples.
•
Functional representation: x ( n )


1 ,

 0 , n

0 elsewhere
•
Sequence representation: x ( n )

{  1 , 1 ,

2 , 1 , 3 ,...}
Hossein Sameti, CE, SUT, Fall 1992 3

Unit Sample
Unit Ramp
Unit Step
Hossein Sameti, CE, SUT, Fall 1992 4

Exponential Signals: x ( n )
 a n for all n
Hossein Sameti, CE, SUT, Fall 1992 5

 Energy of Signals
E





| x ( n ) |
2 vs. Power of Signals
P

N lim
  2 N
1

1 n

N


|
N x ( n ) |
2
Hossein Sameti, CE, SUT, Fall 1992 6

 Periodic vs. aperiodic signals
 A signal is periodic with period N ( N >0) iff x(n+N)=x(n) for all n
 The smallest value of N where this holds is called the fundamental period.
N
Hossein Sameti, CE, SUT, Fall 1992 7

 Symmetric (even) and anti-symmetric (odd) signals:
◦
Even: x(-n) = x(n)
◦
Odd: x(-n) = -x(n)
 Any arbitrary signal can be expressed as a sum of two signal components, one even and the other odd: x e
( n )

1
2
 x ( n )
 x (
 n )

=
+ x o
( n )

1
2
 x ( n )
 x (
 n )
 x ( n )
 x e
( n )
 x o
( n )
Hossein Sameti, CE, SUT, Fall 1992 8

 A discrete-time system is a device that performs some operation on a discrete-time signal.
 A system transforms an input signal x(n) into an output signal y(n) where: .
 Some basic discrete-time systems:
◦
Adders
◦
Constant multipliers
◦
Signal multipliers
◦
Unit delay elements
◦
Unit advance elements
Hossein Sameti, CE, SUT, Fall 1992 9

Hossein Sameti, CE, SUT, Fall 1992 10

Hossein Sameti, CE, SUT, Fall 1992 11

y ( n )
 x ( 2 n )
Source: Stanford
Hossein Sameti, CE, SUT, Fall 1992 12

◦
y(n) = x
1
(n) + x
2
(n)
◦
y(n) = x
1
◦
y(n) = a x(n)
(n) x
2
(n)
Hossein Sameti, CE, SUT, Fall 1992 13

x ( n )



 n ,
0 ,
Solution :

3
 n

3 elsewhere y ( n )

1
3
[ x ( n

1 )
 x ( n )
 x ( n

1 )]
Moving average filter x ( n )

{  0 , 3 , 2 , 1 , 0 ,

1 , 2 , 3 , 0  } y ( 0 )
 y ( n )
1
3
[ x (

1 )
 x ( 0 )


{  0 , 1 ,
5
3
, 2 , 1 , x ( 1 )]

1
3
[ 1

0

1 ]

2
3

, 1 , 2 ,
5
3
, 1 , 0  }
2
3
Hossein Sameti, CE, SUT, Fall 1992 14

x ( n )



 n
0 ,
,

3
 n

3 elsewhere
Solution : y ( n )

[ x ( n )
 x ( n

1 )
 x ( n

2 )

...]
Accumulator x ( n )

{  0 , 3 , 2 , 1 , 0 ,

1 , 2 , 3 , 0  } y ( n )

{  0 , 3 , 5 , 6 , 6

, 7 , 9 , 12 , 12  }
Hossein Sameti, CE, SUT, Fall 1992 15

 Memoryless systems: If the output of the system at an instant n only depends on the input sample at that time
(and not on past or future samples) then the system is called memoryless or static, e.g. y(n)=ax(n)+bx 2 (n)


Otherwise, the system is said to be dynamic or to have memory, e.g. y(n)=x(n)−4x(n−2)
Hossein Sameti, CE, SUT, Fall 1992 16

 In a causal system, the output at any time n only depends on the present and past inputs.
 An example of a causal system: y(n)=F[x(n),x(n−1),x(n− 2),...]
 All other systems are non-causal.
 A subset of non-causal system where the system output, at any time n only depends on future inputs is called anti-causal . y(n)=F[x(n+1),x(n+2),...]
Hossein Sameti, CE, SUT, Fall 1992 17




Unstable systems exhibit erratic and extreme behavior.
BIBO stable systems are those producing a bounded output for every bounded input: x ( n )

M x
   y ( n )

M y
 
Example:
Solution: y ( n ) x ( n )

 y
C

2
( n
( n )

1 )
 x ( n ) Stable or unstable?
Bounded signal y ( 0 )

C , y ( 1 )

C
2
, y ( 2 )

C
4
,..., y ( n )

C
2 n
1

C
  unstable
Hossein Sameti, CE, SUT, Fall 1992 18



Superposition principle: T[ax
1
(n)+bx
2
(n)]=aT[x
1
(n)]+bT[x
2
(n)]
A relaxed linear system with zero input produces a zero output.
Scaling property
Additivity property
Hossein Sameti, CE, SUT, Fall 1992 19



Example:
Solution: y ( n )
 x ( n
2
) y
1
( n )
 x
1
( n
2
)
Linear or non-linear?
y
2
( n )
 x
2
( n
2
) y
3
( n )

T ( a
1 x
1
( n )
 a
2 x
2
( n ))
 a
1 x
1
( n
2
)
 a
2 x
2
( n
2
) a
1 y
1
( n )
 a
2 y
2
( n )
 a
1 x
1
( n
2
)
 a
2 x
2
( n
2
)
Linear!
 Example: y ( n )
 e x ( n )
Useful Hint : In a linear system, zero input results in a zero output!
x ( n )

0
 y ( n )

1
Non-linear!
Hossein Sameti, CE, SUT, Fall 1992 20

 If input-output characteristics of a system do not change with time then it is called time-invariant or shift-invariant. This means that for every input x(n) and every shift k x ( n )
 T y ( n )
 x ( n
 k )
 T y ( n
 k )
Hossein Sameti, CE, SUT, Fall 1992 21

 Time-invariant example: differentiator x ( n ) x ( n

1 )
T
T y ( n )
 x ( n )
 x ( n

1 ) y ( n

1 )
 x ( n

1 )
 x ( n

2 )
 Time-variant example: modulator x ( n )
 T y ( n )
 x ( n ).
Cos (

0
.
n ) x ( n
-
1)
T ¾ x ( n
-
1).
Cos ( w
0
.
n )
¹ y ( n
-
1) y ( n
-
1)
= x ( n
-
1).
Cos ( w
0
.( n
-
1))
Hossein Sameti, CE, SUT, Fall 1992 22

 LTI systems have two important characteristics:
◦
Time invariance: A system T is called time-invariant or shiftinvariant if input-output characteristics of the system do not change with time x ( n )
 T y ( n )
 x ( n
 k )
 T y ( n
 k )
◦
Linearity: A system T is called linear iff
T[ax
1
(n)+bx
2
(n)]=aT[x
1
(n)]+bT[x
2
(n)]
 Why do we care about LTI systems?
◦
Availability of a large collection of mathematical techniques
◦
Many practical systems are either LTI or can be approximated by LTI systems.
Hossein Sameti, CE, SUT, Fall 1992 23

 h (n): the response of the LTI system to the input unit sample

(n), i.e.
h(n)=T (

( n ))
An LTI system is completely characterized by a single impulse response h ( n ).
y ( n )
=
T [ x ( n )]
= k



 x ( k ) h ( n
 k )
 x ( n ) * h ( n )
Response of the system to the input
Convolution unit sample sequence at n=k sum
Hossein Sameti, CE, SUT, Fall 1992 24

y ( n )
 k



 x ( k ) h ( n
 k ) y ( n
0
)
 k



 x ( k ) h ( n
0
 k )
Folding
Shifting and
Multiplying
Repeat for all n
0
Summation
Hossein Sameti, CE, SUT, Fall 1992 25

•
Commutative law: x ( n ) * h ( n )
 h ( n ) * x ( n )
 Distributive law: x ( n ) * [ h
1
( n )]
 h
2
( n )]
 x ( n ) * h
1
( n )
 x ( n ) * h
2
( n )]
Hossein Sameti, CE, SUT, Fall 1992 26

 Associative law:
[ x ( n ) * h
1
( n )] * h
2
( n )
 x ( n ) * [ h
1
( n ) * h
2
( n )]
Hossein Sameti, CE, SUT, Fall 1992 27

h
1
( n )



1
2

 n u ( n )
•
Solution : h
2
( n )



1
4

 n u ( n ) h ( n )
 k



 h
1
( k ) h
2
( n
 k ) v k
( n )
 h
1
( k ) h
2
( n
 k )

1
2 k u ( k

)

1
4 h ( n )

0 , n

0 n
 k u ( n
 k ) h ( n )
 k n 

0
1
2 k
1
4 h ( n )

?
Non-zero for k

0 , n
 k

0 n
 k



1
4

 n k n


0
2 k k n

1


0 ar k  a r

  r n 

1
1


(
1
4
) n
( 2 n

1 
1 ), n

0
Hossein Sameti, CE, SUT, Fall 1992 28

 Remember that for a causal system, the output at any point of time, depends only on the present and past values of the input.
 In the case of an LTI system, causality is translated to a condition on the impulse response. An LTI system is causal iff its impulse response is zero for negative values of n , i.e.
h(n)=0 for n<0


This means that the convolution sum is modified to: y ( n )
 k



0 h ( k ) x ( n
 k )
 k
 n 
 x ( k ) h ( n
 k )
Example: exponential input; h(n)=a n u(n) with |a|<1
Hossein Sameti, CE, SUT, Fall 1992 30

Causality Condition : h ( n )

0 for n

0
Neither necessary nor sufficient condition for all systems, but necessary and sufficient for LTI systems
Proof : y ( n )
 so k



 x ( k ) h ( n
 k )

 k



 x ( n
 k ) h ( k ) k



0 x ( n
 k ) h ( k ) y ( n )
 k



0 x ( n
 k ) h ( k )
But x(n-k) for k>=0 shows the past values of x(n ). So y(n) depends only on the past values of x(n) and the system is causal.
Hossein Sameti, CE, SUT, Fall 1992 31

 Stability: BIBO (bounded-input-bounded-output) stable
|x(n)|<

=> |y(n)|<

 In the case of an LTI system, stability is translated to a condition on the impulse response too. An LTI system is stable iff its impulse response is absolutely summable.
 This implies that the impulse response h(n) goes to zero as n approaches infinity:
S h
 k



 h ( k )
 
Hossein Sameti, CE, SUT, Fall 1992 32

Stability Condition : A linear time-invariant system is stable iff
S h
 k



 h ( k )
  a ) (Sufficien cy) : y [ n ]
 k



 h [ k ] x [ n
 k ]
 k


  h [ k ] x [ n
 k ]

B x k



 h [ k ] b ) (Necessity ) : Let us assume that S h x [ n ]


 h
*
[
 n ] h [

0 n ] h [ n h [ n ]
]


0
0
 
.
Take the input with values y [ 0 ]
 k



 x [
 k ] h [ k ]
 unbounded output
 k


  h [ k ]
2 h [ k ]

S h
Hossein Sameti, CE, SUT, Fall 1992 33

h ( n )
  a b n n
,
, n

0 n

0 a,b=? System Stable
•
Solution : n



 h ( n )
 n



0 a n  n

1


 b n n



0 a n
 
1 b
 n



0 a n 
1
 a
 a
2 
...

1

1 a n

1

  b n  n



1 b
1 n
 
( 1
 

1 b
( 1

1 b
  2 
...)


1
 
 a
1
 b
2
(
1


...)

1 or b

1 )
Hossein Sameti, CE, SUT, Fall 1992 34






LTI systems can be divided into 2 types based on their impulse response:
An FIR system has finite-duration h(n), i.e. h(n) = 0 for n < 0 and n ≥ M. y ( n )

M k



1
0 h ( k ) x ( n
 k )
This means that the output at any time n is simply a weighted linear combination of the most recent M input samples ( FIR has a finite memory of length M).
An IIR system has infinite-duration h(n), so its output based on the convolution formula becomes (causality assumed) y ( n )
 k



0 h ( k ) x ( n
 k )
In this case, the weighted sum involves present and all past input samples thus the IIR system has infinite memory.
Hossein Sameti, CE, SUT, Fall 1992 35

 FIR systems can be readily implemented by their convolution summation (involves additions, multiplications, and a finite number of memory locations).
 IIR systems, however, cannot be practically implemented by convolution as this requires infinite memory locations, multiplications, and additions.
 However, there is a practical and computationally efficient means for implementing a family of IIR systems through the use of difference equations .
Hossein Sameti, CE, SUT, Fall 1992 36

 Cumulative Average System:
( n y (
 n )
1 )
 n y ( n )
1


1 k n


0 x ( k k n



1 x
0
( k )
),
 n
 x ( n )
0 , 1 ,...
 ny ( n

1 )
 x ( n ) y ( n )
 n n

1 y ( n

1 )
 n
1

1 x ( n ) y ( n

1 )

1 n n k

1 

0 x ( k ),
+ Initial Condition n

0 , 1 ,...
Hossein Sameti, CE, SUT, Fall 1992 37

Transmitted waveform
Shifted version of the transmitted waveform + noise
39

 Cross-correlation is an efficient way to measure the degree to which two signals (one template and the other the test signal) are similar to each other.
Shifted version of the template+ noise
Template
 Cross-Correlation is a mathematical operation that resembles convolution. It measures the degree of similarity between two signals.
Hossein Sameti, CE, SUT, Fall 1992 40

Test signal
Cross-Correlation
Machine
Output
Hossein Sameti, CE, SUT, Fall 1992 41

•
The amplitude of each sample in the crosscorrelation signal is a measure of how much the received signal resembles the target signal, at that location .
•
The value of the cross-correlation is maximized when the target signal is aligned with the same features in the received signal.
•
Using cross-correlation to detect a known waveform is frequently called matched filtering .
•
Applications include radar, sonar, biomedical signal processing and digital communications.
Hossein Sameti, CE, SUT, Fall 1992 42

x ( n ) Transmitted/Desired Signal y ( n )
  x ( n

D )
 
( n )
Received/Test Signal
Delayed version of the input
Attenuation factor r xy
( l )
 n



 x ( n ) y ( n
 l )
 n



 x ( n
 l ) y ( n ), l
Additive noise

0 ,


1 ,


2 ,...
• r yx
(l) is thus the folded version of r xy
(l) around l = 0 : r xy
( l )
 r yx
(
 l )
Hossein Sameti, CE, SUT, Fall 1992 43

• Cross-correlation involves the same sequence of steps as in convolution except the folding part, so basically the cross-correlation of two signals involves:
1. Shifting one of the sequences
2. Multiplication of the two sequences
3. Summing over all values of the product
Hossein Sameti, CE, SUT, Fall 1992 44

 The cross-correlation machine and convolution machine are identical, except that in the correlation machine this flip doesn't take place, and the samples run in the normal direction.
r xy
( l )
 x ( l ) * y (
 l )
Cross-correlation is non-commutative.
 Convolution is the relationship between a system's input signal, output signal, and the impulse response. Correlation is a way to detect a known waveform in a noisy background.
 The similar mathematics is only a convenient coincidence.
Hossein Sameti, CE, SUT, Fall 1992 45

r xx

( l )
 n



 x ( n ) x ( n
 l ) n



( n

 x l ) x ( n )
 r xx
(
 l )
Hossein Sameti, CE, SUT, Fall 1992 46

 It can be shown that: r xy
( l )
 r xx
( 0 ) r yy
( 0 )

E x
E y
 For autocorrelation, we thus have: r xx
( l )
 r xx
( 0 )

E x
 This means that autocorrelation of a signal attains its maximum value at zero lag (makes sense as we expect the signal to match itself perfectly at zero lag).
Hossein Sameti, CE, SUT, Fall 1992 47

 If signals are scaled, the shape of the cross-correlation sequence does not change. Only the amplitudes are scaled.



It is often desirable to normalize the auto-correlation and cross-correlation sequences to a range from -1 to 1.
Normalized autocorrelation:  xx
( l )
 r xx
( l ) r xx
( 0 ) r xy
( l )
Normalized cross-correlation:
 xy
( l )
 r xx
( 0 ) r yy
( 0 )
Hossein Sameti, CE, SUT, Fall 1992 48









In this lecture, we learned about:
Representations of discrete time signals and common basic DT signals
Manipulation and representations/diagrams of DT systems
Various classification of DT signals:
Periodic vs. non-periodic, symmetric vs. anti-symmetric
Classifications of DT systems:
◦
Static vs. dynamic, time-invariant vs. time-variant, linear vs. non-linear, causal vs.
◦ non-causal, stable vs. non-stable, FIR vs. IIR
LTI systems and their representation
Convolution for determining response to arbitrary inputs
Cross-correlation
Hossein Sameti, CE, SUT, Fall 1992 49