Intro. to Probability, z-transform,
and Laplace transform
Cheng-Fu Chou
Cheng-Fu Chou, CMLab, CSIE, NTU
Outline
 Probability system
 Conditoinal prob.
 Theorem of total prob.
 Bayes’ theorem
 z-transform
 Laplace transform
P. 2
Cheng-Fu Chou, CMLAB, CSIE, NTU
Probability
 Sample Space (S) which is a collection of objects.
Each is a sample point.
 A family of event S = {A, B, C, …} where a event is a
set of sample points.
 A Probability measure P is an assignment (mapping) of
events defined on S into the set of real numbers.
– P[A] = Probability of event A
P. 3
Cheng-Fu Chou, CMLAB, CSIE, NTU
Properties
 0  P[A]  1
 P[S] = 1
 If A and B are mutually exclusive, then P[A  B] =
P[A] + P[B]
P. 4
Cheng-Fu Chou, CMLAB, CSIE, NTU
Notations
 Event
– A = {w: w satisfies the membership property for
the event A}
 Example
– dice
P. 5
Cheng-Fu Chou, CMLAB, CSIE, NTU
 Ac is the complement of A where { w: w not in A}
 A B = {w: w in A or B or both} = Union
 A B = {w: w in A and B} = Intersection
 0 = empty set
P. 6
Cheng-Fu Chou, CMLAB, CSIE, NTU
 In general
 Let A1, A2, …, An be events
P. 7
Cheng-Fu Chou, CMLAB, CSIE, NTU
Conditional Prob.
 Cond. Prob.
– Constraint sample space
– Scale up
P. 8
Cheng-Fu Chou, CMLAB, CSIE, NTU
Ex. 1
P. 9
Cheng-Fu Chou, CMLAB, CSIE, NTU
Ex. 2
P. 10
Cheng-Fu Chou, CMLAB, CSIE, NTU
Statistical Independent
P. 11
Cheng-Fu Chou, CMLAB, CSIE, NTU
Bayes’ Theorem
 Bayes’ Theorem: we can look at the problem from
another perspective. Assume we know event B has
occurred, but we want to find which mutually
exclusive event has occurred.
P. 12
Cheng-Fu Chou, CMLAB, CSIE, NTU
Ex. 1
 Assume that 15% of job is from I.E., 35% of job is
from E.E., 50% job is from C.S.I.E.. Probability of
read news are 0.01, 0.05, and 0.02 respectively.
– P[a job chosen in random is a Read new jobs]
– P[a randomly chosen job is from EE | it is a read
news job]
P. 13
Cheng-Fu Chou, CMLAB, CSIE, NTU
 P[read] = 0.15*0.01+0.35*0.05+0.5*0.02
 P[read from E.E./read] = (0.35*0.05)/P[read]
P. 14
Cheng-Fu Chou, CMLAB, CSIE, NTU
Ex. 2
 When you walk into a casino, it is a equal prob. for you
to play with honest player or a cheating player. The
prob. for you to lose when playing with a cheating
player is p. Find the
– Prob.[playing with a cheating player | you lost]
P. 15
Cheng-Fu Chou, CMLAB, CSIE, NTU
 P[playing with a cheating player / you lost]
= 0.5*p/(0.5*0.5+0.5*p)
P. 16
Cheng-Fu Chou, CMLAB, CSIE, NTU
Bernoulli trials
 A random experiment that has two outcomes:
“success”: p or “failure” : q (= 1 - p). Now consider a
compound sequence of n independent repetition of
this experiments. This is known as Bernoulli trials.
– What is the prob. of exactly k successes after n
trials ?
– Verification:
P. 17
Cheng-Fu Chou, CMLAB, CSIE, NTU
 n  k nk
(1) p (k )    p q
k 
n
n
 n  k nk
(2) p (k )    p q  ( p  1  p ) n  1
k 0
k 0  k 
P. 18
Cheng-Fu Chou, CMLAB, CSIE, NTU
The Birthday Problem
 In probability theory, the birthday paradox states
that given a group of 23 (or more) randomly chosen
people, the probability is more than 50% that at least
two of them will have the same birthday. For 60 or
more people, the probability is greater than 99%.
– Prove them.
P. 19
Cheng-Fu Chou, CMLAB, CSIE, NTU
Birthday problem (cont.)
 assume there are 365 days in a year
 p is the prob. that no two people in a group of n people
will share a common birthday
– p = (1 – 1/365)(1 – 2/365) …(1 – (n-1)/365)
– p < ½ as n is 23
– P < 0.01 as n is 56
P. 20
Cheng-Fu Chou, CMLAB, CSIE, NTU
Monty Hall Problem
 Based on the American game show “Let’s Make a Deal”
– Suppose you're on a game show, and you're given
the choice of three doors: Behind one door is a car;
behind the others, goats. You pick a door, say No. 1,
and the host, who knows what's behind the doors,
opens another door, say No. 3, which has a goat. He
then says to you, "Do you want to pick door No. 2?"
Is it to your advantage to switch your choice?
P. 21
Cheng-Fu Chou, CMLAB, CSIE, NTU
Random Variables (r.v.)
 We have the probability system (S, S, P)
 R.v. is a variable whose value depends upon the
outcome of the random experiment.
 The outcomes of our random experiment is a w  S.
We associate a real number X(w) to w.
 Thus, the r.v. X(w) is nothing more than a function
defined on the sample space S.
P. 22
Cheng-Fu Chou, CMLAB, CSIE, NTU
Probability distribution function
P. 23
Cheng-Fu Chou, CMLAB, CSIE, NTU
Probability density function
 pdf:
 Different ways to view pdf:
P. 24
Cheng-Fu Chou, CMLAB, CSIE, NTU
Special discrete dist.
 The Bernoulli pmf
 The Binominal pmf: n independent bernoulli trials.
P. 25
Cheng-Fu Chou, CMLAB, CSIE, NTU
Geometric Dist.
 Sequence of bernoulli trials, but we count no. of trials
until First success
– P(i) = P(x=i) = (1-p) i-1 p
– For Geometric dist., it has markov property or it is
memoryless: P[x = i + n | x >n } = P[ x = i]
P. 26
Cheng-Fu Chou, CMLAB, CSIE, NTU
Special Continuous Dist.
 Exponential r.v.
– A continuous r.v. for some l > 0
f(x) = l e –lx if x  0
0
x<0
– For exponential dist., it has Markov property or it
is memoryless
P. 27
Cheng-Fu Chou, CMLAB, CSIE, NTU
Question
 How could we generate a sequence number x1,
x2, …,xn such that xi is a exponential (or any
other specific dist.) r.v. with parameter l?
P. 28
Cheng-Fu Chou, CMLAB, CSIE, NTU
Inverse-transform Technique
 The concept
– For cdf function: r = F(x)
– Generate r from unifrom (0,1)
– Find x such that x = F-1(r)
P. 29
Cheng-Fu Chou, CMLAB, CSIE, NTU
Ans
P. 30
Cheng-Fu Chou, CMLAB, CSIE, NTU
PDF for 2 R.V.
FXY ( x, y )  P[ X  x, Y  y ];
2
d FXY ( x, y )
f XY ( x, y ) 
dxdy
 Marginal Density Function
P[ x1  X  x2 , y1  Y  y2 ]  
x2
x1

y2
y1
f XY ( xy )dydx
Independent : FXY ( xy )  FX ( x) FY ( y )  Prob[X  x]Prob[Y  y]
FXY ( x, y )
FX |Y ( x | y ) 
 Prob[ X  x | Y  y ]
FY ( y )
P. 31
Cheng-Fu Chou, CMLAB, CSIE, NTU
Function of Random Variable
 Func. of r.v.
Y  g( X )
FY ( y)  P[Y  y ]  P[{w : g ( X (w))  y}]
 One important r.v. is where Xi are independent
n
Y   Xi
i 1
P. 32
Cheng-Fu Chou, CMLAB, CSIE, NTU
Y = X1 + X2
 PDF
FY ( y )  P[Y  y ]  P[ X 1  X 2  y ]


y  x2
 f ( x x )dx dx 
 [ 
f ( x )dx ] f ( x )dx
 F ( y  x ) f ( x )dx 
 f ( y)  f ( y  x ) f ( x )dx

X1 X 2
 


1 2
1
2
y  x2
X1

1
1
X2
2
2

 pdf

2
X2
2
2

Y
 Convolution
X1

X1
2
X2
2
2
fY ( y)  f X1 ( y)  f X 2 ( y)
P. 33
Cheng-Fu Chou, CMLAB, CSIE, NTU
Convolution Ex.
• f(n)
• 2/3 as n = 1
• 1/3 as n = 2
• g(n)
• 1/2 as n = 1
• 1/2 as n = 2
 h(n) = f(n)g(n) = ?
P. 34
Cheng-Fu Chou, CMLAB, CSIE, NTU
Ex. (cont.)
 h(0) = f(0)g(0) = 0
 h(1) = f(1)g(0) + f(0)g(1) = 0
 h(2) = f(2)g(0) + f(1)g(1) + f(0)g(2) = 1/3
 h(3) = f(3)g(0)+f(2)g(1)+f(1)g(2)+f(0)g(3) = ½
 h(4) = f(4)g(0)+f(3)g(1)+f(2)g(2)+f(1)g(3)+f(0)g(4) =
1/6
P. 35
Cheng-Fu Chou, CMLAB, CSIE, NTU
z transform
 Consider a func. Of discrete time fn s.t.
– fn  0 for n = 0, 1, 2, …
– fn = 0 for n = -1, -2, …
–
f n  F ( z )where F ( z )   n 0 f n z

n
P. 36
Cheng-Fu Chou, CMLAB, CSIE, NTU
Examples
 Ex1:
f n  A n forn  0,1, 2,...


n 0
n 0
 F ( z )   A n z n  A ( Z ) n


  z
 Ex2:
n
Convolution property f n  gn   f nk gk
k 0
P. 37
Cheng-Fu Chou, CMLAB, CSIE, NTU
Convolution Property

n
f n  g n   [ f n  k g k ]z n
n 0 k 0

n
 [ f n  k g k ]z n  k z k
n 0 k 0


  [ g k z k ] f n k z n k
k 0 n k


k 0
nk


k 0
m0
 g k z k  f n  k z n  k
  g k z k  f m z m 
G ( z ) F ( z )
P. 38
Cheng-Fu Chou, CMLAB, CSIE, NTU
Properties of z transform

1. f n  F ( z )   f n z n
n 0
2.af n  bg n  aF ( z )  bG ( z )
3.a n f n  F (az )

4. f n 1  
n 0
1 
1
n 1
f n 1 z   f n 1 z  [ F ( z )  f 0 ]
z n 0
z
n


5. f n 1   f n 1 z  z  f n 1 z n 1  zF ( z )
n
n 0
n 0


6.nf n   nf n z  
n
n 0
n 0
dz n
d
fn
z  z F (z)
dz
dz
7. f n  g n  F ( z )G ( z )
P. 39
Cheng-Fu Chou, CMLAB, CSIE, NTU
Properties (cont.)
8. f n  f n 1  F ( z )  zF ( z )  (1  z ) F ( z )

n

n

9. f k , n  0,1, 2,...   f k z   f k z z
n
k 0
n 0 k 0
k
k 0 n  k
nk
F ( z)

1 z


10. f n (n is a parameter of f n ) 
F ( z)
a
a

11.F (1)   f n
n 0
12. F (0)  f 0
P. 40
Cheng-Fu Chou, CMLAB, CSIE, NTU
z Transform pair
 1n  0
1.U n  
 F ( z)  1
0

otherwise

2.U n  k  z k

3. n  1 forn  0,1,...   z n 
n0
4. n  k
zk

1 z

5. A   A n z n 
n
n 0
6.n n 
7.n 
1
1 z
A
1 z
z
(1   z ) 2
z
(1  z ) 2

1
zn
8.  F ( z )    e z
n!
n 0 n !
Read and derive Table I.1 and I.2
P. 41
Cheng-Fu Chou, CMLAB, CSIE, NTU
z transform : difference equation
 Ex:
6 g n  5 g n 1  g n  2
1 n
 6( ) n  2,3, 4,...
5
6
g 0  0; g1 
5
P. 42
Cheng-Fu Chou, CMLAB, CSIE, NTU
z-transform and moment

G( z)   f n z n
n 0
dG ( z ) 
dG ( z )
n 1
  nf n z 
dz
dz
n 0

z 1
  nf n  X
n 0
d 2G ( z ) 
n2

n
(
n

1)
f
z


n
2
dz
n 0
d 2G ( z )
dz 2

2
2

(
n

n
)
f

X
X

z 1
n
n 0
P. 43
Cheng-Fu Chou, CMLAB, CSIE, NTU
Laplace transform
 Def:
1.F *( s )  


 st
f (t )e dt ;
2. f (t )  F *( s )
 Ex 1.
 Ex 2.
 Ae at t  0
f (t )  
0otherwise
1 t  0
 (t )  
0 t  0
P. 44
Cheng-Fu Chou, CMLAB, CSIE, NTU
Convolution
 f(t) and g(t) take on non-zero values for t 0
f (t )  g (t )  


f (t  x) g ( x)dx
P. 45
Cheng-Fu Chou, CMLAB, CSIE, NTU
Properties
1. af (t )  bg (t )  aF *( s)  bG *( s)
t
2. f ( )  aF *(as)
a
3. f (t  a)  e  as F *( s)
4. e  at f (t )  F *( s  a )
d
5. tf (t )   F *( s )
ds
n
d
F *( s )
n
n
6. t f (t )  (1)
ds n

f (t )
7.
  F *( s1)ds1
s1 s
t
P. 46
Cheng-Fu Chou, CMLAB, CSIE, NTU
Properties (cont.)
df (t )
8.
 ?( sF *( s)  f (0 ))
dt
d n f (t )
n
n 1

n 1

9.

s
F
*(
s
)

s
f
(0
)

...

f
(0
)
n
dt
t
F *( s)
10.  f (t )dt 

s
t
t
F *( s)
n
11.  ... f (t )(dt ) 


sn
P. 47
Cheng-Fu Chou, CMLAB, CSIE, NTU
Differential eq.
 Find f(t)
d 2 f (t ) 6 f (t )

 9 f (t )  2t ;
2
dt
dt

df
(0
)

f (0 ) 
 0;
dt
P. 48
Cheng-Fu Chou, CMLAB, CSIE, NTU
Random Sum
Given Y  X 1  X 2  ...  X N~ , find Y
~
X i are i.i.d. N is a discretenon - negat iver.v.

~
~
Y * (s)   Y * (s | N  n)P [N  n]
n 0
~
Y * (s | N  n)  [X * (s)]n

~
Y * (s)   [X * (s)] P [N  n]
n
n 0

~
Note that N ( z )   P[ N  n]z n
n 0
Relatez with [X * (s)]n
Y * ( s )  N [ X * ( s )]
P. 49
Cheng-Fu Chou, CMLAB, CSIE, NTU
Ex.
Y  X 1  X 2  ...  X N~
X i are i.i.d. exponential dist ribut ed,
~
N is geomet rically dist ribut ed.
(1) Find E[Y].
(2) Whatis var[Y]
P. 50
Cheng-Fu Chou, CMLAB, CSIE, NTU