Random
Ross
Chapter
Variables
2 (a) and Dis.

Random variable X: A function that associates a real number with
each element (sample point) in the sample space S.

An example: Toss a fair die
– Sample space S = {top, bottom, front, back, left, right}.
– Random variable X: X(top)=1, X(bottom)=2, X(front)=3,
X
top
→
bottom →
front
→
back
→
left
→
right
→
X(back)=4, X(left) = 5, X(right) = 6.
1
2
3
4
5
6
1
Random Variable (RV)


A function from a sample space to the real
numbers
X :Ω → R
Ex. Let X be the number of heads when two
coins are tossed.
(T , T)
(H , T)
(T , H)
(H, H)
[0,1]
R
Ω
X=0
1/4
X=1
1/2
X=2
1/4
2
Random
An
RV isVariables
an “Uncertain
and Dis.
Quantity”

Why are random variables used?
– Easy to manipulate: Toss a die twice (with faces numbered as 1, 2,
3, 4, 5, and 6):

X1: The output of the first toss;

X2: The output of the second toss;

X = X1+X2 gives the total output.

How to interpret (12 – X) ?
– Simple and elegant!
– Effective!!!
Once the die are tossed, there is no more randomness or uncertainty.
For example, x = 9 is the realized value of the RV X.
3
Battery Lifetime
4
Section 2.2 Probability distribution functions
R.V. and Distribution function
Read from Page 24 (after ex 2.5) to 30

Probability distribution function (PDF) (Cumulative
distribution function (cdf)): Toss a die:
–
–
–
–
–
–
–
–
–
–
P(X ≤ 0) = P(empty) = 0;
P(X ≤ 1) = P({top}) = 1/6;
P(X ≤ 2) = P({top, bottom}) = 2/6;
P(X ≤ 3) = P({top, bottom, front}) = 3/6;
P(X ≤ 4) = P({top, bottom, front, back}) = 4/6;
P(X ≤ 5) = P({top, bottom, front, back, left}) = 5/6;
P(X ≤ 6) = P({top, bottom, front, back, left, right}) = 6/6 = 1;
P(X ≤ 7) = P({top, bottom, front, back, left, right}) = 6/6 = 1;
P(X ≤ –1.5) = 0; P(X ≤ 3.5) = 3/6; P(X ≤ 88.8) = 6/6. (Why?)
P(X ≤ x) can be found for any real number x.
5
R.V. and Distribution function

Cumulative distribution function (CDF) of X: F(x)
Definition: F(x) = P(X ≤ x), –∞ < x < ∞.
– Probability: P(a < X ≤ b) = F(b) – F(a), for a ≤ b.
Example: Toss a die:
0,
𝐹𝐹 𝑥𝑥 =
1/6,
2/6,
3/6,
4/6,
5/6,
6/6,
𝑖𝑖𝑖𝑖 − ∞ < 𝑥𝑥 < 1;
𝑖𝑖𝑖𝑖 1 ≤ 𝑥𝑥 < 2;
𝑖𝑖𝑖𝑖 2 ≤ 𝑥𝑥 < 3;
𝑖𝑖𝑖𝑖 3 ≤ 𝑥𝑥 < 4;
𝑖𝑖𝑖𝑖 4 ≤ 𝑥𝑥 < 5;
𝑖𝑖𝑖𝑖 5 ≤ 𝑥𝑥 < 6;
𝑖𝑖𝑖𝑖 6 ≤ 𝑥𝑥 < ∞.
F(x)
1
5/6
4/6
3/6
2/6
1/6
0
1
2
3
4
5
6
x
– P(1.8 < X ≤ 3.5) = F(3.5) – F(1.8) = 3/6 – 1/6 = 2/6. (Intuitively?)
6
R.V. and Distribution function

Random variable X takes only countable number of values.
– Probability mass (PDF):
f(x) = P(X=x) ≥ 0 and
– CDF:
F ( x) = P( X ≤ x) = ∑ f (t ), − ∞ < x < ∞.
t≤x

Random variable X takes more than countable many values.
– Cumulative distribution function F(x) = P(X ≤ x), for any real x.
– F(x) is nondecreasing between 0 and 1.
– Density function (PDF): f(x) = F’(x), for any real x. (derivative)
1
7
Probability Distributions

Discrete distributions (Probability mass
function)
PX ( X = xi ) = PX ( xi ) = pi = p( xi ),
∑ P ( X = x ) = 1,
i
X
i
1 ≥ PX ( xi ) ≥ 0
j
PX (i ≤ X ≤ j ) = ∑ PX ( X = k )
k =i
Ex. Let X be the number of heads when two
coins are tossed.
P( X = 0) = 1 / 4
P( X = 1) = 2 / 4
P( X = 2) = 1 / 4
8
Section 2.2 Discrete Distributions

Discrete distributions (*)
–
–
–
–
Bernoulli trials
Binomial distribution
Geometric distribution
Poisson distribution
9

Bernoulli trial
– Bernoulli trial: success w.p. p or failure w.p. q=1– p.
– Bernoulli process: A series of independent Bernoulli trials. The
outcome of the nth Bernoulli trial is Xn.
– Xn = 1 if the nth trial is a success and Xn = 0 if the nth trial is a
failure: P{Xn=0}=1–p and P{Xn=1}=p.
– F(x) = P{Xn ≤ x} ?
– E[Xn] = 0*(1–p) + 1*p = p.
– Var(Xn) = σ2 = 02*(1–p) + 12*p – p2= p(1–p).
– Example: Flip a coin: {head, tail}. X: head → 1; tail → 0.
10

Binomial distribution X
– Let X be the number of successes among n Bernoulli trials with
probability of success p. Then X has a binomial distribution with
parameters (n, p).
n i
n!
n −i


P( X = i ) =   p (1 − p ) =
p i (1 − p ) n −i , 0 ≤ i ≤ n.
i!(n − i )!
i 
– F(x) = P{X ≤ x} ?
– Intuition? X = X1 + X2 + … + Xn.
– E[X] = np (intuition?) and Var(X) = np(1–p). (Intuition?)
11
12
Look at examples 2.8 and 2.9
13

Geometric distribution X
– Bernoulli process with success probability p. The outcome of
the nth Bernoulli trial is Xn (1 for success and 0 for failure)
– X is the number of trials until the first success occurs.
P(X = n) = p(1–p)n –1, n=1, 2, 3, …
(2.4)
– F(x) = P{X ≤ x} ?
– Intuition? X = min {n: Xn = 1}
– E[X] = 1/p (intuitive explanation?)
– Var(X) = (1–p)/p2.
– P(X = n) decays geometrically with decay rate 1–p. (How?)
14
15
Survival example:
An individual, near the end of life, may die at the
end of each year with probability 0.2.
a. What is the probability of surviving at least 5
more years? (Ans: ~ .41)
b. What is the expected number of additional life
years? (Mean = 5 yrs. But std dev ~ 4.5 yrs.)
16

Poisson distribution with parameter λ
λn
P( X = n) = exp{−λ} , n ≥ 0.
n!
– n = 0, 1, 2, 3, …
– F(x) = P{X ≤ x} ?
– E[X] = Var(X) = λ. (A useful feature to identify Poisson
distribution)
– Let X1, X2, …, Xk be k independent Poisson distributions
with parameter λ1, λ2, …, λk. Then X = X1+X2+ …+Xk has
a Poisson distribution with parameter λ1+λ2+…+λk.
(Closure property)
17
Look at examples 2.11 and 2.12
18
Section 2.3 Continuous Distributions

Continuous distributions (*)
–
–
–
–
Uniform distribution
Exponential distribution
Normal distribution
Gamma distribution
19
Probability Distributions

Continuous Distributions (Probability density
function) ∞
b
f X ( x) ≥ 0,
∫f
−∞
X
( x)dx = 1, P(a ≤ X ≤ b) = ∫ f X ( x)dx,
a
Uniform Distribution
1 /(b − a) if a ≤ x ≤ b
f X ( x) = 
otherwise
 0
1/(b-a)
a
b
20
Cumulative Distribution Function

FX ( x) = P( X ≤ x)
1. Fx is a monotonic function. Why?
2. P (a ≤ X ≤ b) = FX (b) − FX (a )
3. P( x ≤ X ≤ x + dx) = f X ( x)dx
or
dFX ( x)
= f x ( x)
dx
For the Exponential distribution (See slide 25)
λe − λt if t ≥ 0
f X (t ) = 
 0 otherwise
FX (t ) = 1 − e − λt
21
R.V. and Dis. (continued)

An example
– F(x)=0, x<0; F(x) = (x2+x)/6, 0≤x≤2;
F(x)=1, x>2.
f(x) = (2x+1)/6, 0≤x≤2;
f(x)=0, x>2.
f(x)=0, x<0;
P(0.5<X<1.5) = F(1.5) – F(0.5) = 3.75/6 – 0.75/6 = 3/6.
1
2
22

X has the Uniform distribution on [a, b] :
– PDF:
if − ∞ < x < a, b < x < ∞;
0,

f ( x) =  1
 b − a , if a ≤ x ≤ b.
– CDF:
if − ∞ < x < a;
0,
x −a

F ( x) = 
, if a ≤ x ≤ b;
b − a
if b < x < ∞.
1,
– E[X] = (a+b)/2.
– Var(X) = (b–a)2/12.
– Transformation: U = (X – a)/(b – a) is the uniform
distribution on [0, 1]. Thus, X = a + (b–a)U.
23

Normal distribution X
– Normal(µ, σ), where µ is the mean and σ is the standard
deviation.
 ( x − µ )2 
1
exp−
– Density function f ( x) =
, − ∞ < x < ∞.
2
2σ 
2π σ

– No explicit formula for CDF F(x): F ( x) = P ( X < x) = ∫
– “Bell-shaped curve”
x
−∞
f ( x)dx.
– Symmetric about the mean µ
– Approximation to the Binomial: Z = (X – np)/(npq)0.5.
24
25
Ross Chapter 2 (b)
Section 2.5 Jointly Distributed RVs
26
Marginal Distribution Functions
Marginal PMF
Marginal Density
Function
Let’s see the corresponding Marginal CDFs
27
R.V. and
Joint
Probabilities
Dis. (Joint dis.)

Consider random variables X, Y, Z, …
– Discrete case: P(X=x, Y=y) = f(x, y, z).
– Example: P(X=0, Y=5) = 0.2, P(X=0, Y=10) = 0.35,
P(X=1, Y=5) = 0.4, P(X=1, Y=10) = 0.05.
Marginal dis.: P(X=0) = 0.55, P(X=1) = 0.45;
P(Y=5) = 0.6,
P(Y=10) = 0.4. (Independent?)
28
Table 3.1 Joint Probability
Distribution for Example 3.14
Example 3.14
p(x,y)
3 - 29
R.V. and Dis. (Joint dis.)

Continuous joint distribution: X, Y, Z, …
– Density function:
f(x, y, z) ≥ 0 with
∞ ∞ ∞
∫ ∫ ∫ f ( x, y, z )dxdydz = 1.
− ∞− ∞− ∞
– Distribution function:
F(x, y, z) = P(X≤x, Y≤y, Z≤z) =
z y x
∫ ∫ ∫ f ( x, y, z )dxdydz.
− ∞− ∞− ∞
– In general, consider a set (region) A, then
P (( X , Y , Z ) ∈ A) = ∫∫∫ f ( x, y, z )dxdydz.
A
– Marginal distribution:
P( X < x) =
x ∞ ∞
∫ ∫ ∫ f ( x, y, z )dzdydx.
− ∞− ∞− ∞
30
Mathematical Expectation (example)

Example:
follow.
The joint density function of X and Y is given as
f ( x, y ) = 2( x + 2 y ) / 7, 0 < x < 1, 1 < y < 2.
Then
2
f X ( x) = ∫ 2( x + 2 y )dy / 7 = 2 x / 7 + 6 / 7, 0 < x < 1.
1
1
fY ( y ) = ∫ 2( x + 2 y )dx / 7 = 1 / 7 + 4 y / 7, 1 < y < 2.
0
31
R.V. and Dis. (Joint dis.)

Continuous Joint Distributions (continued)
– Example: f(x, y, z) = kxy2z, for 0<x<1, 0<y<1, 0<z<2.
1) Determine the constant k. (k=3)
2) Find P(X<0.25, Y>0.5, 1<Z<2).
(A = {(x, y, z): x<0.25, y>0.5, 1<z<2})
This example is simple since X, Y, and Z are independent.
– Example: f(x, y) = k, for x2 + y2 = 1.
1) Determine the constant k (=1/(2π)) ?
2) P(0<X, 0<Y) = (0.25) ?
32
R.V. and Distribution function

Independence of Random Variables
 Definition: Random variables X and Y are independent iff
P(X≤x, Y≤y) = P(X≤x)P(Y≤y), for all real x and y.
– For continuous cases, random variables X1, X2, …, and Xn are
independent iff
f(x1, x2, …, xn) = f(x1) f(x2) … f(xn).
– Why is independence useful? Computing probabilities!!! For
instance,
P( X 1 < x1 , X 2 < x2 ) =
x1
∫f
1
−∞
x2
(t )dt ∫ f 2 (t )dt.
−∞
33
R.V. and Distribution function
– Example: Consider two light bulbs who’s lifetime density
functions are f1(x) = 2exp{-2x} and f2(x) = 5exp{-5x}. The two
light bulbs are independent. What is the probability that both
light bulbs will be working for more than 20 hours (x=20)?
34
Chpt 2.4 Expectation p.g. 34-41 (you can skip
example
Mathematical
2.22 and 2.27)
Expectation

Mathematical Expectations:
– Mean, variance, covariance, etc.
– Standard deviation, correlation coefficient, etc.

Why? Sometimes probability distribution is not available. Sometimes
probability distribution is not convenient for a decision.
– Example Consider two business investment plans:
Plan A: investment = 1 million;
Return = 0, w.p. 0.9;
15 millions, w.p. 0.1.
Plan B: investment = 1 million;
Return = 0.8 millions, w.p. 0.7;
2 millions, w.p. 0.3.
The mean and the variance can be helpful in this case.
35
Mathematical Expectation

Mathematical expectation as the weighted average
– Toss a fair die (with six faces numbered by 1, 2, 3, 4, 5, 6) for a
million times, what is the average output?
 Average output (definition) = (X1+X2+…+X1000000)/1000000.
 Intuitively, Average output
≈ 1(1/6)+2(1/6)+3(1/6)+4(1/6)+5(1/6)+6(1/6)
= 7*6/(2*6) = 3.5.
– Formally, the mean of the output X is defined by
E[X]
= 1P(X=1)+2P(X=2)+3P(X=3)+4P(X=4)+5P(X=5)+6P(X=6)
= 1(1/6)+2(1/6)+3(1/6)+4(1/6)+5(1/6)+6(1/6)
= 3.5.
36
Discrete Case
37
Continuous Case
38
Var(X)=
Chapters 2-7.39
Expectation of a Function
Check example 2.24.
40
Mathematical Expectation
Variance

Variance and standard deviation of r.v. X

Example: Toss a fair die
– µX = 3.5.
σ X 2 = E[( X − µ X ) 2 ]
1
1
1
+ (2 − 3.5) 2 + (3 − 3.5) 2
6
6
6
1
1
1
+ (4 − 3.5) 2 + (5 − 3.5) 2 + (6 − 3.5) 2 = 2.91667
6
6
6
= (1 − 3.5) 2
41
= 2.9166
42
Expectation of Jointly Distributed Variables
Mathematical
Expectation
now revisit chapter
2.5 pg 43-48(example)
and ex 2.34.
43

Example:
The joint density function of X and Y is given as
follow. Find the expected value of Z = X/Y3 + X2Y.
f ( x, y ) = 2( x + 2 y ) / 7, 0 < x < 1, 1 < y < 2.
1 2
E ( Z ) = ∫ ∫ ( x / y 3 + x 2 y ) f ( x, y )dxdy
0 1
1 2
= ∫ ∫ ( x / y 3 + x 2 y )2( x + 2 y ) / 7dxdy
0 1
1 2
 x2
2x
3
2 22
= ∫ ∫  3 + x y + 2 + 2 x y  dxdy.
y
y
7
0 1
Can we find the marginal distributions of X and Y?
44
Mathematical Expectation

Expected value of linear combinations of r.v.
E[aX + bY + c] = aE[ X ] + bE[Y ] + c.

Expected value of the product XY when X and Y are
independent
E[ XY ] = E[ X ]E[Y ].
45
46
47
Mathematical Expectation

Coefficient of variation (cv) of X
cv( X ) = σ X / µ X

Covariance of r.v.s X and Y
σ XY = E[( X − µ X )(Y − µY )] = E[ XY ] − µ X µY .

Correlation Coefficient (–1≤ρXY≤1)
ρ XY

σ XY
=
.
σ Xσ Y
If X and Y are independent, σXY = ρXY = 0 (why?)
48
Other properties of Expectation
49
50
Convolution pg. 52-54 (only examples 2.36
and 2.37)
51


Ex,
What is the distribution of sum of two iid exponential
RVs with rate 5/hr?
52
Moment Generating Function pg 58-63
53
Moment Generating Function pg 58-63
54
55
Sum of RVs
56