11.1
Random Variables:
In order to develop an efficient analysis and characterization of random events, it is
useful to have events correspond to a number. Thus, a random variable RV is a
function that maps the sample space (outcomes) into real numbers.
Let ζ be an outcome or point in the sample space, the random variable associated
with this outcome is denoted by:
x = X (ζ )
In many engineering applications the random outcome is already a number, such as
• a voltage measured from a noisy channel
• the power in the vibrations measure from a bridge
• the number of times a communications channel is accessed in a given minute
• the consumption of electric power in an industrial plant
Example consider a 4 coin toss experiment.
1. Describe several random variables that can be defined from this experiment.
2. Choose one of the random variables and plot the probability distribution for each
value of the random variable.
11.2
Probability Distribution Functions:
The probabilities associated with each of the values of the random variable can be
described in terms of the probability distribution function (pdf) or the cumulative
distribution function (cdf).
Probability Distribution Function (pdf)
For a discrete random variable the pdf is a function that assigns a probability to
each value of the random variable f ( x ) = P ( X = x) . For a continuous random
variable the pdf is a function that assigns a probability to intervals of values for the
random variable. ∫ f ( x )dx = P (a < X < b)
X
b
a
X
Example: Create a pdf for the RV determined by the sum of two dice faces.
Example: Sketch the pdf for a uniformly distributed RV, X, with values ranging
from 0 to 10. Determine P(3<X<5.5), P(8.75>X ), P(12>X>9), P(X=3), and P(X=23).
The Cumulative Distribution Function (cdf)
The cdf is a function that assigns a probability to the random variable
F ( x ) = P ( X ≤ x) .
Example: Do the previous 2 examples for the cdf instead of the pdf.
X
11.3
Properties and Relationships for the cdf and pdf
cdf's limiting values:
lim F ( x ) = 0 , lim F ( x ) = 1
X
x →−∞
x →∞
X
Continuity (cdf's are continuous from the right):
lim F ( x ) = F ( x )
X
x → x0+
X
0
cdf are monotonic and nondecreasing functions:
0≤ F ( x ) ≤ F ( x ) ≤ 1 ∀ x ≤ x
X
1
X
2
1
2
Probability of RV occurring in interval [a, b]:
P( a ≤ X ≤ b) = F (b) − F ( a )
X
X
For continuous RV's:
dF ( x )
F ( x ) = ∫ f ( λ )dλ
f ( x) =
dx
x
X
X
X
−∞
X
pdf's nonnegativity constraint:
11.4
f ( x) ≥ 0
X
∀ x
pdf's unit area:
1 = ∫ f ( λ )dλ
∞
−∞
X
Probability of RV occurring in interval [a, b]:
P (a ≤ X ≤ b) = F (b) − F (a ) = ∫ f ( x )dx
b
X
X
a
Example: Consider a RV, X, with a Rayleigh pdf:
x
2x
f ( x) =
exp⎛⎜ − ⎞⎟ u( x )
⎝ σ ⎠
σ
a) Find the cdf for X. (show F ( x ) = (1 − exp( − ( x / σ ) ) )u( x ) )
b) Find the probability of X < 1, given σ=1. (show P(X < 1) = .6321)
c) Find the probability of 1.5 <X < 2, given σ=1. (show P(1.5<X < 1) = 0.0878)
d) Find the probability of X > 4, given σ=1. (show P(X> 4) = 0.0183)
2
X
2
2
2
X
11.5
e) Plot the pdf and cdf when σ=1. Show graphically part c on the pdf, and part b on
the cdf.
Rayleigh pdf
0.9
» x = [0:.05:6]; % Create RV axis
» y = (2*x / 1).*exp(-(x / 1).^2);
» plot(x, y)
» title('Rayleigh pdf')
» xlabel('X')
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
» cy = cumsum(y)*(.05);
» plot(x,cy)
» title('Rayleigh cdf')
» xlabel('X')
0
1
2
3
X
4
5
6
Rayleigh cdf
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0
1
2
3
X
4
5
6
11.6
The Uniform RV:
This distribution is used to model a continuum of equally likely events mapped into
a finite range.
pdf
⎧⎪ 1
,
for a ≤ x ≤ b
f ( x ) = ⎨b − a
⎪⎩ 0,
otherwise
cdf
0,
for a ≤ x
⎧
⎪x − a
for a < x ≤ b
F ( x) = ⎨
,
⎪b − a
for x > b
1,
⎩
X
X
The expected value (mean) for this distribution is defined as:
b+a
E[ X ] = ∫ xf ( x )dx =
2
The expected value of the dispersion about the mean is the variance defined by:
(b − a )
E[( X − E ( X )) ] = ∫ ( x − E ( X ) ) f ( x )dx =
12
∞
−∞
X
∞
2
−∞
2
2
X
11.7
Gaussian RV:
This distribution models a continuum of events with a central tendency. This
distribution occurs most often in random natural phenomena. It is sometimes
referred to as the normal RV.
pdf
1
( x − m) ⎞
f ( x) =
exp⎛⎜ −
⎟
⎝
2πσ
2σ ⎠
cdf: Since this is a useful function and no closed form of the integration exits, a
special function will be defined:
1
⎛ ( λ − m) ⎞ dλ = 1 ⎛ 1 + erf ⎛ x − m⎞ ⎞
F ( x) =
⎟
⎜
⎟⎟
⎜
∫ exp⎜⎝ −
⎝ 2σ ⎠ ⎠
2πσ
2σ ⎠
2⎝
2
where erf ( y ) =
∫ exp( − u )du
2
X
2
2
x
X
2
−∞
y
2
π
0
The expected value (mean) for this distribution is defined as:
E[ X ] = ∫ xf ( x )dx = m
∞
−∞
X
The expected value of the dispersion about the mean is the variance defined by:
E[( X − E ( X )) ] = ∫ ( x − E ( X ) ) f ( x )dx = σ
∞
2
−∞
2
2
X
11.8
Example: Given a digital communications channel that corrupts transmitted signals
with zero-mean additive Gaussian Noise. The average noise power is 1 Watt.
Assume a binary 0 is sent with a zero voltage level and a binary 1 is sent with a 5
volt level. The receiver sets a 2.5 volt threshold to detect transmitted zeros and ones
such that any voltage greater than 2.5 is accepted as a 1 and any voltage less than
2.5 is accepted as a 0. Compute the probabilities of detection and error for all
possible transmit-receive combinations.
Sketch distributions for the received signals for the cases of the transmitted 1 and 0.
» x = [-5:.05:10]; % Create RV axis
» y1 = exp(-(x-5).^2/(2*1))/sqrt(2*pi*1);
» y0 = exp(-(x-0).^2/(2*1))/sqrt(2*pi*1);
» plot(x,y0,'r',x,y1,'b')
» title('Distribution of received signals')
» xlabel('X')
% Probability of detecting a zero given
a one was sent.
» p0g1 = .5*(1+erf((2.5-5)/(sqrt(2)*1)))
p0g1 = 0.0062
% Probability of detecting a zero given
a zero was sent.
» p0g0 = .5*(1+erf((2.5-0)/(sqrt(2)*1)))
p0g0 = 0.9938
Distribution of received signals
0.4
0.35
Received
binary 0
Received
binary 1
0.3
0.25
0.2
0.15
0.1
0.05
0
-5
0
X
5
10
11.9
Binomial Distribution
This distribution models n independent trails with binary outcomes (i.e. success or
failure). The RV is k, the number of successes where p is the probability of success
for each outcome.
pdf- Probability Mass Function
⎛ n⎞
⎛ n⎞
for k = 0,1,2,.... or f ( x ) = ∑ ⎜ ⎟ p (1 − p) δ ( x − k )
P ( X = k ) = ⎜ ⎟ p (1 − p)
⎝ k⎠
⎝ k⎠
cdf
⎛ n⎞
F ( x ) = ∑ ⎜ ⎟ p (1 − p)
⎝ k⎠
The expected value (mean) for this distribution is defined as:
E[ X ] = ∫ xf ( x )dx = np
n
n− k
k
X
n− k
k =0
n− k
k
X
k
k≤x
∞
−∞
X
The expected value of the dispersion about the mean is the variance defined by:
E[( X − E ( X )) ] = ∫ ( x − E ( X ) ) f ( x ) dx = np(1 − p)
∞
2
−∞
2
X
Example: A signal measured from a certain process has a probability of .001 of
crossing a threshold. For 20 measured signals, find the probability that at least one
of them crosses the threshold. (Show P= 0.0198)
11.10
Poisson Distribution
The Poisson distribution can model the number of occurrences, k, of a random event
over a given interval T.
pdf - Probability Mass Function
(λ T )
P( X = k ) =
exp( − λ T ) for k = 0,1,...
k!
where λ is the rate of occurrence.
cdf
( λ T ) exp( −λ T )
F ( x) = ∑
k!
The expected value (mean) for this distribution is defined as:
E[ K ] = ∑ kP ( k ) = λ T
k
k
X
k≤x
∞
k =0
The expected value of the dispersion about the mean is the variance defined by:
E[( K − E ( K )) ] = ∑ ( k − E ( K ) ) P ( k ) = λ T
2
∞
k =0
2
11.11
Closely related to the Poisson Distribution is the exponential distribution, which can
be used to model the interval, w, between two random events occurring with rate λ.
pdf f ( w) = λ exp( − λ w) u( w)
cdf F ( w) = (1 − exp( − λ w))u( w)
Example: In the afternoon, a certain computer receives information packets at a
rate of 30 per minute. Find the probability at least a .01 second interval will occur
between packets in the afternoon. (Show P= 0.9950)
W
W
Hypergeometric Distribution
This distribution models sampling n items from a batch of N items without
replacement where K items are defective (or different).
Probability Mass Function
⎛ K⎞ ⎛ N − K⎞
⎜ ⎟⎜
⎟
⎜ ⎟⎜
⎟
⎝ k ⎠⎝ n − k ⎠
, k = 0,1,..., n
P( X = k ) =
⎛ N⎞
⎜ ⎟
⎜ ⎟
⎝ n⎠