CIS 2033 based on
Dekking et al. A Modern Introduction to Probability and Statistics, 2007
Instructor Longin Jan Latecki
Chapter 7: Expectation and variance
The expectation of a discrete random variable X taking the values a1,
a2, . . . and with probability mass function p is the number:
E[ X ]   ai P(X  ai )   ai p(ai )
i
i
We also call E[X] the expected value or mean of X. Since the expectation is
determined by the probability distribution of X only, we also speak of the
expectation or mean of the distribution.
Expected values
of discrete random variable
Example
• Let X be the discrete random variable that takes the
values 1, 2, 4, 8, and 16, each with probability 1/5.
Compute the expectation of X.
E[ X ]   ai P(X  ai )   ai p(ai )
i
i
Bernoulli Distribution
• Let X have Bernoulli distribution with the
probability of success p.
E[ X ]   ai P(X  ai )   ai p(ai )
i
i
E ( X )   xP( x )  (0)(1  p )  (1)( p )  p
x
Var( X )   ( x  p ) 2 P ( x )  (0  p ) 2 (1  p )  (1  p ) 2 ( p )
x
 p(1  p )( p  1  p )  p(1  p )
Var ( X )  E[( X  E[ X ])2 ]
Binomial Distribution
• Let X have Binomial distribution with the probability of
success p and the number of trails n.
• Computing the expectation of X directly leads to a
complicated formula, but we can use the fact that X can
be represented as the sum of n independent Bernoulli
variables:
X  X 1  ...  X n
E ( X )  E ( X 1  ...  X n)  E ( X 1 )  ...  E ( X n)  p  ...  p  np
Var( X )  Var( X 1  ...  X n)  Var( X 1 )  ...  Var( X n)  np(1  p )
Note: We do not need the independence assumption for the expected value,
since it is a linear function of RVs, but we need it for variance.
Geometric Distribution
• Let X have Geometric distribution with the
probability of success p.
E[ X ]   ai P(X  ai )   ai p(ai )
i

1
1
E ( X )   x(1  p ) p 
p
2
(1  (1  p ))
p
x 1
1 p
Var( X )  2
p
x
We skip the derivation of variance.
i
The expectation of a continuous random variable X with probability density
function f is the number

E[ X ] 
 xf ( x)dx

We also call E[X] the expected value or mean of X. Note that E[X] is indeed
the center of gravity of the mass distribution described by the function f:


E[ X ] 
 xf ( x)dx 

 xf ( x)dx
-

 f ( x)dx
-
Expected values
of continuous random variable
Uniform U(a,b)
• Let X be uniform U(a, b). Then f(x)= 1/(b-a) for x in [a, b]
and zero outside this interval.

E[ X ] 
 xf ( x)dx


1
1 b2  a 2 b  a
1 2 1 
E ( X )   xf ( x )dx   x
dx   x



ba
2
 2 b  a a 2 b  a

a
b
(b  a ) 2
Var( X )  E ( X )  E ( X ) 
12
2
2
b
The EXPECTATION of a GEOMETRIC DISTRIBUTION. Let X have a
geometric distribution with parameter p; then

E[ X ]   kp(1  p)k 1 
k 1
1
p
The EXPECTATION of an EXPONENTIAL DISTRIBUTION. Let X have an
exponential distribution with parameter λ; then

E[ X ]   xe dx 
 x
0
1

The EXPECTATION of a NORMAL DISTRIBUTION. Let X be an N(μ, σ2)
distributed random variable; then

1 X 
 (
1
E[ X ]   x
e 2
  2

)2
dx  
The CHANGE-OF-VARIABLE FORMULA. Let X be a random variable, and
let g : R → R be a function.
If X is discrete, taking the values a1, a2, . . . , then
E[ g ( X )]   g (ai )P(X  ai )
i
If X is continuous, with probability density function f, then

E[ g ( X )] 
 g ( x ) f ( x ) dx

Example: Let X have a Ber(p) distribution. Compute E(2X).
The variance Var(X) of a random variable X is the number
Var ( X )  E[( X  E[ X ])2 ]
Standard deviation:
std  Var ( X )
Variance of a normal distribution. Let X be an N(μ, σ2) distributed random
variable. Then
1 x

 (
1
2
Var ( X )   ( x   )
e 2
 2


)2
dx   2
Variance of an EXPONENTIAL DISTRIBUTION. Let X have an exponential
distribution with parameter λ; then
Var ( X ) 
1
2
An alternative expression for the variance. For any random variable X,
Var ( X )  E[ X 2 ]  E[ X ]2
E[ X 2 ] is called the second moment of X.
We can derive this equation from:


Var ( X )  E[( X  E[ X ]) ]   ( x  E[ X ] ) f ( x )dx   ( x 2  2 xE[ X ]  ( E[ X ])2 ) f ( x )dx
2
2





 x f ( x)dx  2E[ X ]  xf ( x)dx  ( E[ X ])
2

 ( E[ X 2 ])  ( E[ X ])2
2




f ( x )dx  E[ X 2 ]  2( E[ X ])2  ( E[ X ])2
Example.
Let X takes the values 2, 3, and 4 with probabilities 0.1, 0.7, and 0.2.
We can compute that E[X]= 3.1.
Var ( X )  E[( X  E[ X ])2 ]
Var ( X )  E[ X 2 ]  E[ X ]2
Expectation and variance under change of units. For any random variable X
and any real numbers r and s,
E[rX  s]  rE[ X ]  s
and
Var (rX  s)  r 2Var ( X )