Review of Probability Concepts
Prepared by Vera Tabakova, East Carolina University

B.1 Random Variables

B.2 Probability Distributions

B.3 Joint, Marginal and Conditional Probability
Distributions

B.4 Properties of Probability Distributions

B.5 Some Important Probability Distributions
Principles of Econometrics, 3rd Edition
Slide B-2

A random variable is a variable whose value is unknown until it is
observed.

A discrete random variable can take only a limited, or countable,
number of values.

A continuous random variable can take any value on an interval.
Principles of Econometrics, 3rd Edition
Slide B-3

The probability of an event is its “limiting relative frequency,” or the
proportion of time it occurs in the long-run.

The probability density function (pdf) for a discrete random
variable indicates the probability of each possible value occurring.
f ( x)  P  X  x 
f ( x1 )  f ( x2 ) 
Principles of Econometrics, 3rd Edition
 f ( xn )  1
Slide B-4
Principles of Econometrics, 3rd Edition
Slide B-5
Figure B.1 College Employment Probabilities
Principles of Econometrics, 3rd Edition
Slide B-6

The cumulative distribution function (cdf) is an alternative way to
represent probabilities. The cdf of the random variable X, denoted
F(x), gives the probability that X is less than or equal to a specific
value x.
F  x  P  X  x
Principles of Econometrics, 3rd Edition
Slide B-7
Principles of Econometrics, 3rd Edition
Slide B-8

For example, a binomial random variable X is the number of
successes in n independent trials of identical experiments with
probability of success p.
n x
P  X  x   f  x     p (1  p ) n x
 x
 n
n!
where n!  n  (n  1 n  2 
 
 x  x! n  x !
Principles of Econometrics, 3rd Edition
(B.1)
 21
Slide B-9
Figure B.2 PDF of a continuous random variable
Principles of Econometrics, 3rd Edition
Slide B-10
P  20  X  40  
40
 f  x  dx .355
20
P  X  x 
x
 f  t  dt  F  x 

P  20  X  40   F (40)  F (20)  .649  .294  .355
Principles of Econometrics, 3rd Edition
Slide B-11
1
2

X 
3
4
high school diploma or less
some college
four year college degree
advanced degree
0 if had no money earnings in 2002
Y 
1 if had positive money earnings in 2002
Principles of Econometrics, 3rd Edition
Slide B-12
 f  x, y   1
x
Principles of Econometrics, 3rd Edition
y
Slide B-13
f X ( x )   f ( x, y )
for each value X can take
y
fY ( y )   f ( x, y )
(B.2)
for each value Y can take
x
4
fY  y    f  x, y 
y  0,1
x 1
fY 1  .19  .06  .04  .02  .31
Principles of Econometrics, 3rd Edition
Slide B-14
Principles of Econometrics, 3rd Edition
Slide B-15
P(Y  y, X  x) f ( x, y )
f ( y | x)  P(Y  y | X  x) 

P  X  x
f X ( x)
Principles of Econometrics, 3rd Edition
y
f  y | X  3
0
.04/.18=.22
1
.14/.18=.78
(B.3)
Slide B-16

Two random variables are statistically independent if the conditional
probability that Y = y given that X = x, is the same as the
unconditional probability that Y = y.
P  Y  y | X  x   P Y  y 
(B.4)
f ( x, y)
f ( y | x) 
 fY ( y )
f X ( x)
(B.5)
f ( x, y )  f X ( x ) f Y ( y )
(B.6)
Principles of Econometrics, 3rd Edition
Slide B-17
Principles of Econometrics, 3rd Edition
Slide B-18
Principles of Econometrics, 3rd Edition
Slide B-19
Principles of Econometrics, 3rd Edition
Slide B-20
Principles of Econometrics, 3rd Edition
Slide B-21

B.4.1
Mean, median and mode
E[ X ]  x1P  X  x1   x2 P  X  x2  

 xn P  X  xn 
(B.7)
For a discrete random variable the expected value is:
  E[ X ]  x1 f ( x1 )  x2 f ( x2 ) 
n
  xi f ( xi )   xf ( x)
i 1
Principles of Econometrics, 3rd Edition
 xn f ( xn )
(B.8)
x
Slide B-22

For a continuous random variable the expected value is:
  EX  

 xf  x dx

The mean has a flaw as a measure of the center of a probability
distribution in that it can be pulled by extreme values.
Principles of Econometrics, 3rd Edition
Slide B-23

For a continuous distribution the median of X is the value m such that
P  X  m   P( X  m)  .5

In symmetric distributions, like the familiar “bell-shaped curve” of
the normal distribution, the mean and median are equal.

The mode is the value of X at which the pdf is highest.
Principles of Econometrics, 3rd Edition
Slide B-24
E[ g ( X )]   g ( x) f ( x)
(B.9)
x
E  aX   aE  X 
(B.10)
E  g  X    g  x  f  x   axf  x   a xf  x   aE  X 
Principles of Econometrics, 3rd Edition
Slide B-25
E  aX  b  aE  X   b
(B.11)
E  g1  X   g2  X   E  g1  X   E  g2  X 

(B.12)
The variance of a discrete or continuous random variable X is the
expected value of
g  X    X  E  X  
Principles of Econometrics, 3rd Edition
2
Slide B-26

The variance of a random variable is important in characterizing the
scale of measurement, and the spread of the probability distribution.

Algebraically, letting E(X) = μ,
var( X )    E  X    E[ X 2 ]  2
2
Principles of Econometrics, 3rd Edition
2
(B.13)
Slide B-27
Figure B.3 Distributions with different variances
Principles of Econometrics, 3rd Edition
Slide B-28
var(aX  b)  a2 var( X )
(B.14)
var(aX  b)  E  aX  b  E  aX  b    E  aX  b  a  b 
2
2
 E  a  X      a E  X     a 2 var  X 
2
Principles of Econometrics, 3rd Edition
2
2
Slide B-29
3

E  X   

skewness   3

4

E  X   

kurtosis  
4
Principles of Econometrics, 3rd Edition
Slide B-30
E[ g ( X , Y )]   g ( x, y ) f ( x, y )
x
(B.15)
y
E  X  Y   E ( X )  E (Y )
(B.16)
E  X  Y     x  y  f  x, y   xf  x, y    yf  x, y 
x
y
x
y
x
y
  x  f  x, y    y  f  x, y    xf  x    yf  y 
x
y
y
x
x
y
 E  X   E Y 
Principles of Econometrics, 3rd Edition
Slide B-31
E (aX  bY  c)  aE ( X )  bE (Y )  c
(B.17)
E  XY   E  g  X , Y     xyf  x, y   xyf  x  f  y 
x
y
x
y
  xf  x   yf  y   E  X  E Y  if X and Y are independent.
x
y
g ( X , Y )  ( X   X )(Y  Y )
Principles of Econometrics, 3rd Edition
(B.18)
Slide B-32
Figure B.4 Correlated data
Principles of Econometrics, 3rd Edition
Slide B-33
cov( X , Y )   XY  E  X   X Y  Y   E  XY    X Y
cov  X , Y 
 XY


var( X ) var(Y )  X Y

(B.19)
(B.20)
If X and Y are independent random variables then the covariance and
correlation between them are zero. The converse of this relationship is
not true.
Principles of Econometrics, 3rd Edition
Slide B-34

If a and b are constants then:
var  aX  bY   a 2 var( X )  b2 var(Y )  2ab cov( X , Y )
(B.21)
var  X  Y   var( X )  var(Y )  2cov( X , Y )
(B.22)
var  X  Y   var( X )  var(Y )  2cov( X , Y )
(B.23)
Principles of Econometrics, 3rd Edition
Slide B-35

If X and Y are independent then:
var  aX  bY   a 2 var( X )  b2 var(Y )
(B.24)
var  X  Y   var( X )  var(Y )
(B.25)
var  X  Y  Z   var  X   var Y   var  Z 
Principles of Econometrics, 3rd Edition
Slide B-36
4
E  X    xf  x   1 .1   2  .2    3  .3   4  .4   3   X
x 1
  E  X  X 
2
X
2
2
2
2
2







 1  3  .1   2  3  .2   3  3  .3   4  3  .4 

 
 
 

  4  .1  1 .2    0  .3  1 .4 
1
Principles of Econometrics, 3rd Edition
Slide B-37

B.5.1
The Normal Distribution

If X is a normally distributed random variable with mean μ and
variance σ2, it can be symbolized as X ~ N  , 2  .
 ( x  )2 
f ( x) 
exp 
,

2
22
 2

1
Principles of Econometrics, 3rd Edition
  x  
(B.26)
Slide B-38
Figure B.5a Normal Probability Density Functions with Means μ and Variance 1
Principles of Econometrics, 3rd Edition
Slide B-39
Figure B.5b Normal Probability Density Functions with Mean 0 and Variance σ2
Principles of Econometrics, 3rd Edition
Slide B-40

A standard normal random variable is one that has a normal
probability density function with mean 0 and variance 1.
X 
Z
~ N (0,1)


(B.27)
The cdf for the standardized normal variable Z is
( z )  P  Z  z  .
Principles of Econometrics, 3rd Edition
Slide B-41
a 
 X  a 

 a  
P[ X  a]  P 


P
Z






 
 
 
  
(B.28)
a 
 X  a 

 a  
P[ X  a]  P 


P
Z


1





 
 
 
  
(B.29)
b 
a 
 b 
 a  
P[a  X  b]  P 
Z 
 
  


 
 
  
  
(B.30)
Principles of Econometrics, 3rd Edition
Slide B-42

A weighted sum of normal random variables has a normal
distribution.
X 1 ~ N  1 , 12 
X 2 ~ N   2 , 22 
Y  a1 X1  a2 X 2 ~ N  Y  a11  a22 , Y2  a1212  a2222  2a1a212 
Principles of Econometrics, 3rd Edition
(B.27)
Slide B-43
V  Z12  Z22 
 Zm2 ~ (2m)
(B32)
E[V ]  E (2m )   m
2

var[V ]  var ( m )   2m
Principles of Econometrics, 3rd Edition
(B.33)
Slide B-44
Figure B.6 The chi-square distribution
Principles of Econometrics, 3rd Edition
Slide B-45

A “t” random variable (no upper case) is formed by dividing a
standard normal random variable Z ~ N  0,1 by the square root of an
independent chi-square random variable, V ~ (2m) , that has been
divided by its degrees of freedom m.
Z
t
V
Principles of Econometrics, 3rd Edition
~ t( m )
(B.34)
m
Slide B-46
Figure B.7 The standard normal and t(3) probability density functions
Principles of Econometrics, 3rd Edition
Slide B-47

An F random variable is formed by the ratio of two independent chisquare random variables that have been divided by their degrees of
freedom.
V1 m1
F
~ F( m1 ,m2 )
V2 m2
Principles of Econometrics, 3rd Edition
(B.35)
Slide B-48
Figure B.8 The probability density function of an F random variable
Principles of Econometrics, 3rd Edition
Slide B-49















binary variable
binomial random variable
cdf
chi-square distribution
conditional pdf
conditional probability
continuous random variable
correlation
covariance
cumulative distribution function
degrees of freedom
discrete random variable
expected value
experiment
F-distribution
Principles of Econometrics, 3rd Edition














joint probability density function
marginal distribution
mean
median
mode
normal distribution
pdf
probability
probability density function
random variable
standard deviation
standard normal distribution
statistical independence
variance
Slide B-50