CIS 2033 based on
Dekking et al. A Modern Introduction to Probability and Statistics, 2007
Instructor Longin Jan Latecki
Chapter 7: Expectation and variance
The expectation of a discrete random variable X taking the values a1,
a2, . . . and with probability mass function p is the number:
E[ X ] ai P(X ai ) ai p(ai )
i
i
We also call E[X] the expected value or mean of X. Since the expectation is
determined by the probability distribution of X only, we also speak of the
expectation or mean of the distribution.
Expected values
of discrete random variable
Example
• Let X be the discrete random variable that takes the
values 1, 2, 4, 8, and 16, each with probability 1/5.
Compute the expectation of X.
E[ X ] ai P(X ai ) ai p(ai )
i
i
Bernoulli Distribution
• Let X have Bernoulli distribution with the
probability of success p.
E[ X ] ai P(X ai ) ai p(ai )
i
i
E ( X ) xP( x ) (0)(1 p ) (1)( p ) p
x
Var( X ) ( x p ) 2 P ( x ) (0 p ) 2 (1 p ) (1 p ) 2 ( p )
x
p(1 p )( p 1 p ) p(1 p )
Var ( X ) E[( X E[ X ])2 ]
Binomial Distribution
• Let X have Binomial distribution with the probability of
success p and the number of trails n.
• Computing the expectation of X directly leads to a
complicated formula, but we can use the fact that X can
be represented as the sum of n independent Bernoulli
variables:
X X 1 ... X n
E ( X ) E ( X 1 ... X n) E ( X 1 ) ... E ( X n) p ... p np
Var( X ) Var( X 1 ... X n) Var( X 1 ) ... Var( X n) np(1 p )
Note: We do not need the independence assumption for the expected value,
since it is a linear function of RVs, but we need it for variance.
Geometric Distribution
• Let X have Geometric distribution with the
probability of success p.
E[ X ] ai P(X ai ) ai p(ai )
i
1
1
E ( X ) x(1 p ) p
p
2
(1 (1 p ))
p
x 1
1 p
Var( X ) 2
p
x
We skip the derivation of variance.
i
The expectation of a continuous random variable X with probability density
function f is the number
E[ X ]
xf ( x)dx
We also call E[X] the expected value or mean of X. Note that E[X] is indeed
the center of gravity of the mass distribution described by the function f:
E[ X ]
xf ( x)dx
xf ( x)dx
-
f ( x)dx
-
Expected values
of continuous random variable
Uniform U(a,b)
• Let X be uniform U(a, b). Then f(x)= 1/(b-a) for x in [a, b]
and zero outside this interval.
E[ X ]
xf ( x)dx
1
1 b2 a 2 b a
1 2 1
E ( X ) xf ( x )dx x
dx x
ba
2
2 b a a 2 b a
a
b
(b a ) 2
Var( X ) E ( X ) E ( X )
12
2
2
b
The EXPECTATION of a GEOMETRIC DISTRIBUTION. Let X have a
geometric distribution with parameter p; then
E[ X ] kp(1 p)k 1
k 1
1
p
The EXPECTATION of an EXPONENTIAL DISTRIBUTION. Let X have an
exponential distribution with parameter λ; then
E[ X ] xe dx
x
0
1
The EXPECTATION of a NORMAL DISTRIBUTION. Let X be an N(μ, σ2)
distributed random variable; then
1 X
(
1
E[ X ] x
e 2
2
)2
dx
The CHANGE-OF-VARIABLE FORMULA. Let X be a random variable, and
let g : R → R be a function.
If X is discrete, taking the values a1, a2, . . . , then
E[ g ( X )] g (ai )P(X ai )
i
If X is continuous, with probability density function f, then
E[ g ( X )]
g ( x ) f ( x ) dx
Example: Let X have a Ber(p) distribution. Compute E(2X).
The variance Var(X) of a random variable X is the number
Var ( X ) E[( X E[ X ])2 ]
Standard deviation:
std Var ( X )
Variance of a normal distribution. Let X be an N(μ, σ2) distributed random
variable. Then
1 x
(
1
2
Var ( X ) ( x )
e 2
2
)2
dx 2
Variance of an EXPONENTIAL DISTRIBUTION. Let X have an exponential
distribution with parameter λ; then
Var ( X )
1
2
An alternative expression for the variance. For any random variable X,
Var ( X ) E[ X 2 ] E[ X ]2
E[ X 2 ] is called the second moment of X.
We can derive this equation from:
Var ( X ) E[( X E[ X ]) ] ( x E[ X ] ) f ( x )dx ( x 2 2 xE[ X ] ( E[ X ])2 ) f ( x )dx
2
2
x f ( x)dx 2E[ X ] xf ( x)dx ( E[ X ])
2
( E[ X 2 ]) ( E[ X ])2
2
f ( x )dx E[ X 2 ] 2( E[ X ])2 ( E[ X ])2
Example.
Let X takes the values 2, 3, and 4 with probabilities 0.1, 0.7, and 0.2.
We can compute that E[X]= 3.1.
Var ( X ) E[( X E[ X ])2 ]
Var ( X ) E[ X 2 ] E[ X ]2
Expectation and variance under change of units. For any random variable X
and any real numbers r and s,
E[rX s] rE[ X ] s
and
Var (rX s) r 2Var ( X )
0
