ECE 450 - Lecture 4
• Overview
– Simulation & MATLAB
– Random Variables: Concept and Definition
– Cumulative Distribution Functions (CDF’s)
– Examples & Properties
– Probability Distribution Functions (pdf’s)
1
Random Number Generators
• Computers are inherently deterministic, and can’t generate
random numbers
• Instead they use various deterministic algorithms to generate
pseudo-random numbers
– Such algorithms, called random number generators
(RNG’s) start with a seed, that determines the sequence
of pseudo-random numbers generated
– Any time the RNG starts with same seed, the same
pseudo-random numbers will result
– To avoid getting the same sequence on repeated runs,
the “seed” is often a function of the computer clock time.
2
Computer Simulation
• A simulation is a computer program that replicates
– the behavior of some device or system; or
– the execution of some experiment.
• Simulation gives us insight into the operation of the
actual system (or replicates the experiment) without
requiring us to actually construct the system or execute
the experiment
• Hence, simulation saves money!
• A simulation program you already know: PSPICE
– Maybe you also know: Simulink
3
Using MATLAB
to Generate Random Integers [MATLAB Help]
randi: Pseudorandom integers from a uniform discrete distribution.
R = randi(IMAX,N) returns an N-by-N matrix containing pseudorandom
integer values drawn from the discrete uniform distribution on
1:IMAX.
randi(IMAX,M,N) or randi(IMAX,[M,N]) returns an M-by-N matrix.
R = randi([IMIN,IMAX],...) returns an array containing integer values
drawn from the discrete uniform distribution on IMIN:IMAX.
Example: 2 rows, 4 columns of random integers between 0 and 10 (incl.)
>> my_rand = randi( [0, 10], [2, 4])
my_rand =
8 1 6 3
9 10 1 6
4
Example with MATLAB m-file
• Say we want to simulate the toss of 2 dice; let X be the #
showing on one die, and let Y be the # showing on the
other die;
• Also say that Z is the sum of the 2 #’s showing: Z = X + Y
• Suppose that we want to execute the experiment 1000
times, and find the “probability” that Z = 7.
% program dice
X = randi([1, 6], [1, 1000]);
Y = randi([1, 6], [1, 1000]);
Z = X + Y;
flag7 = (Z == 7);
count7 = sum(flag7)
prob = count7/1000
5
Course Overview
– Where Are We Now?
I.
Basic Probability Rules, Definitions
II. Random Variables
III. Random Processes – An Introduction
6
The “Random Variable” Concept
• Say we perform an experiment (say E), the (numerical)
outcome of which we will call X.
– X is a variable, in the sense that it can take on
(possibly) many values;
– X is random, in the sense that we cannot predict (with
any certainty) what the outcome will be, a priori.
• Random variables allow us to give numerical descriptions
of the outcome of an experiment.
• A random variable is continuous if it can take any value
over a continuous range
7
Random Variables: Definition
• A random variable is a mapping (or a function)
– the domain is the sample space of an experiment;
– the range is some subset of the set of real numbers.
• Random variables assign a numerical value to every
possible experimental outcome.
• Example: say we toss 2 coins, and define a RV which
counts how many heads appear:
S
HH
TH
HT
TT
0
1
2
3
R
Outcome
TT
TH
HT
HH
RV X
8
RV’s: Another Example
• Experiment: Toss a single die
• Sample space S = {_________________}
• (Arbitrarily) Define RV X such that X is five times the #
showing on the die
• (Arbitrarily) Define RV Y such that Y is 3 more than the #
showing on the die
Die
1
2
X
5
10
Y
4
5
3
4
5
6
9
RV’s: Continued Example
• To answer probability questions about the RV, “back up”, to
get an equivalent question about the experimental outcome
• Examples: Pr(X = 10) = Pr(die = 2) = _____
Pr(Y = 8) = Pr(die = 5) = _____;
Pr(Y = 5.5) = ______
Pr(X > 10) = Pr(die = 3, 4, 5, or 6) = _____
Pr(Y  6) = Pr(Y = 4, 5, or 6) = Pr(die = ___________) = __
Die
1
2
3
4
5
6
X
5
10
15
20
25
30
Y
4
5
6
7
8
9
10
Probability Distribution Function
(also called Cumulative Distribution Function, CDF)
• Defn: The cumulative distribution function CDF for the
random variable X is
FX(x) = Pr(X  x)
the RV
• Example:
(Note: It is a probability.)
a real #
Die
1
2
3
4
5
6
X
5
10
15
20
25
30
x
FX(x)
-1
Pr(X  -1) = ____
4
Pr(X  4) = ____
FX(x)
x
11
CDF Example, continued
Die
1
2
3
4
5
6
X
5
10
15
20
25
30
x
5
7
FX(x) = Pr(X  x)
FX(x)
FX(x)
1
9
10
11.2
14.999
15
19
20
0
5
10
15
20
25
30
x
12
CDF Properties
0  FX(x)  1
(because it’s a probability)
FX(-) = 0, FX() = 1
FX(x) is monotone non-decreasing
Pr(a < X  b) = FX(b) – FX(a)
1.
2.
3.
4.
x
Typical cdf’s
5.
a
FX(x)
b
FX(x)
1
1
.6
.3
0
x
(for a continuous RV)
0
0
1
3 4
(for a discrete RV)
x
13
Special Case 1 - The Uniform RV
• Consider a RV with CDF
0
xa
1
x>b
linear from 0 to 1 on (a, b)
FX(x) =
1
This RV is said to be
uniformly distributed on
(a, b);
x
a
b
Notation:
U(a, b)
Example of a continuous
RV
14
Example: Say X is U(0, 2)
FX(x)
Some Conclusions
1
Pr(X  1) = Fx(1) = ______
Pr(X  ½) = _____ = _____
x
0
2
Pr(X  1.5) = _____ = ____
Pr(X  27) = ______ = ______
Pr(X > ½) = 1 – Pr(X  ½) = 1 - Fx(½) = _____ (corollary 1)
Pr(X  (0, 1]) = Pr(0 < X  1) = Fx(1) – Fx(0) = ___ - ___ = ___
Pr(X = 1) = ______;
Pr(X  (0, 1)) = _____
15
Discrete & Continuous RV’s
• Discrete RV’s take only a countable # of values, and have
stair-case CDF’s
– The probability of the RV taking any specific value is
given by the size of the jump in the CDF at that value
• Continuous RV’s take on a continuum of values, and have
continuous CDF’s
– The probability of the RV taking any specific value is 0
(also the size of the (non-existent) jump in the CDF at
that value)
– Uniform RV’s are continuous RV’s
16
Continuous RV, “Generic”
FX(x)
1
General,
arbitrary shape
a
x0
b
x
• Say RV X is continuous on (a, b)
• Pr(X = x0) = ______
17
CDF for Continuous RV – An Example
• Experiment: say we are testing diodes, starting at t = 0;
• Define RV T = time to failure
• Say FT(t) = Pr(T  t) = 0,
1 – exp(-mt),
t<0
else
• Pr(diode fails between* times t = a and t = b)
m is a parameter of
the distribution
= Pr(diode fails before t = b) – Pr(diode fails before t = a)
= FT(b) – FT(a) = (1 – exp(-mb) – (1 – exp(-ma)
= exp(-ma) – exp(-mb)
* Don’t worry about the endpoints, since the RV is continuous.
18
Example, continued
1
FT(t), with m = 2
1
0.9
0.8
.9817
.8647
0.7
For m = 2, assume that t is
measured in months; find
the probability that the
diode failure occurs
between 1 and 2 months.
0.6
0.5
0.4
0.3

0.2
0.1
0
0
0
0.5
1
1
1.5
2
2
2.5
3
3
Pr(failure between 1 and 2 months)
= FT(2) – FT(1) = [1 – exp(-2(2)] – [1 – exp(-2(1)]
= exp(-2(1)) – exp(-2(2))
= .117
19
More CDF Properties, & Classification
• Pr(X = a) = Da (size of jump in FX(x))
• FX(x) is right-continuous (“holes” filled in on top, open
on bottom)
F (x)
X
• FX(x)
 continuous  X is a continuous RV
 stair-case  X is a discrete RV
 else  X is a mixed-type RV
1
x
 FX(x) is monotone non-decreasing; i.e.,
x1 < x2  FX(x1)  FX(x2)
20
CDF Review
21
Probability Density Functions (PDF’s)
• Consider a continuous RV, X
• Recall FX(x)  Pr(X  x), CDF
• Define fX(x)  d FX ( x ), the pdf for the RV X
dx
x
 FX(x) =  fx ( t ) dt
(

a
• Note: Pr(a < X  b) = Pr(X  b) – Pr(X < a)
= FX(b) – FX(a) =
=
b
 fx ( t ) dt
a
b
 fx ( t ) dt

-
]
b
x
a
 fx ( t ) dt

22
PDF’s
fX(x)
a
Pr(a < X  b)
b
• Notes
 fX(x)  0
x
fX(x)
for all x
fX(x)
 Pr(x < X  x + dx)  fx(x)dx

x x+dx

dx
  f X ( x ) dx  1
x
23
CDF, PDF Facts
• FX(x) and fX(x) are each complete descriptions of the RV X
• Knowing one, we can always find the other
– Hence they are “information equivalent”
• Knowing either one enables us to answer all probability
questions
24
Example: Say X is U(2, 5)
• Then
FX(x)
1
Slope = rise/run = 1/3
2
• Thus,
5
x
dF/dx, on (2,5)
fX(x): uniform
1/3
Check: Area = 1
2
5
x
• Pr(1 < X < 3) = _____
25
Generalization: PDF’s for Discrete RV’s
• Recall for Discrete RV’s, FX(x) is stair-case
• Example:
FX(x)
d/dx
fX(x)
1
½
0
1
2
x
0
1
2
x
26
Dirac Delta Function: Review
• Dirac delta:
– Area = 1 (shown in parentheses)
– Amplitude = 
d(x)
(1)
x
0
• Shifted Delta:
d(x-a)
(1)
• Sifting Property of Delta Functions
0
a
x

 f ( x ) d( x  a) dx  _____

• Note: In Probability, the area of the delta function at a (in
the pdf) is the height of the jump in FX(x), or Pr(X = a).
27
Specific Example: Discrete RV
• Experiment: Transmit 3 bits over a noisy channel, where
errors occur independently from bit-to-bit, with probability
0.1. If RV X is the # of errors appearing in a 3-bit word
on reception, find the pdf and cdf for X.
• Solution: RV X can take on any of the 3 values: 0, 1, 2,
or 3, with the following probabilities:
X = xk
0
1
Pr(X = xk)
3 0
  (.1) (.9)3  .729
0
3 1 2
  (.1) (.9)  .243
 1
28
Example, continued
X = xk
Pr(X = xk)
2
3 2
  (.1) (.9)1  .027
 2
3
 .001
S = 1 (?)

fX(x)
FX(x)
1
1
.972
.729
(not to scale)
0
1
2
3
0
1
2
3
29
Properties of PDF’s
1. The area under the entire pdf is 1:
2. To use a pdf to calculate a probability:
Pr(a < X  b) = ________________
3. PDF’s are never negative: fX(x)  0
x
4. FX(x) =
 fX ( t ) dt

30