Introduction to Econometrics
Review of Probability
Based on slides made by C. Dougherty
Introduction to Econometrics
Review of Probability
1. Probability Distribution
(a simple example)
SMU Classification: Restricted
PROBABILITY DISTRIBUTION EXAMPLE: X IS THE SUM OF TWO DICE
red
1
2
3
4
5
6
This sequence provides an example of a discrete random variable. Suppose that you have
a red die which, when thrown, takes the numbers from 1 to 6 with equal probability.
1
SMU Classification: Restricted
PROBABILITY DISTRIBUTION EXAMPLE: X IS THE SUM OF TWO DICE
red
green
1
2
3
4
5
6
1
2
3
4
5
6
Suppose that you also have a green die that can take the numbers from 1 to 6 with equal
probability.
2
SMU Classification: Restricted
PROBABILITY DISTRIBUTION EXAMPLE: X IS THE SUM OF TWO DICE
red
green
1
2
3
4
5
6
1
2
3
4
5
6
We will define a random variable X as the sum of the numbers when the dice are thrown.
3
SMU Classification: Restricted
PROBABILITY DISTRIBUTION EXAMPLE: X IS THE SUM OF TWO DICE
red
green
1
2
3
4
5
6
1
2
3
4
5
6
10
For example, if the red die is 4 and the green one is 6, X is equal to 10.
4
SMU Classification: Restricted
PROBABILITY DISTRIBUTION EXAMPLE: X IS THE SUM OF TWO DICE
red
green
1
2
3
4
5
6
1
2
3
4
5
7
6
Similarly, if the red die is 2 and the green one is 5, X is equal to 7.
5
SMU Classification: Restricted
PROBABILITY DISTRIBUTION EXAMPLE: X IS THE SUM OF TWO DICE
red
green
1
2
3
4
5
6
1
2
3
4
5
6
7
2
3
4
5
6
7
8
3
4
5
6
7
8
9
4
5
6
7
8
9
10
5
6
7
8
9
10
11
6
7
8
9
10
11
12
The table shows all the possible outcomes.
6
SMU Classification: Restricted
PROBABILITY DISTRIBUTION EXAMPLE: X IS THE SUM OF TWO DICE
red
green
1
2
3
4
5
6
1
2
3
4
5
6
7
2
3
4
5
6
7
8
3
4
5
6
7
8
9
4
5
6
7
8
9
10
5
6
7
8
9
10
11
6
7
8
9
10
11
12
X
f
p
2
3
4
5
6
7
8
9
10
11
12
1
2
3
4
5
6
5
4
3
2
1
1/36
2/36
3/36
4/36
5/36
6/36
5/36
4/36
3/36
2/36
1/36
If you look at the table, you can see that X can be any of the numbers from 2 to 12.
7
SMU Classification: Restricted
PROBABILITY DISTRIBUTION EXAMPLE: X IS THE SUM OF TWO DICE
red
green
1
2
3
4
5
6
1
2
3
4
5
6
7
2
3
4
5
6
7
8
3
4
5
6
7
8
9
4
5
6
7
8
9
10
5
6
7
8
9
10
11
6
7
8
9
10
11
12
X
f
p
2
3
4
5
6
7
8
9
10
11
12
1
2
3
4
5
6
5
4
3
2
1
1/36
2/36
3/36
4/36
5/36
6/36
5/36
4/36
3/36
2/36
1/36
We will now define f, the frequencies associated with the possible values of X.
8
SMU Classification: Restricted
PROBABILITY DISTRIBUTION EXAMPLE: X IS THE SUM OF TWO DICE
red
green
1
2
3
4
5
6
1
2
3
4
5
6
7
2
3
4
5
6
7
8
3
4
5
6
7
8
9
4
5
6
7
8
9
10
5
6
7
8
9
10
11
6
7
8
9
10
11
12
X
f
p
2
3
4
5
6
7
8
9
10
11
12
1
2
3
4
5
6
5
4
3
2
1
1/36
2/36
3/36
4/36
5/36
6/36
5/36
4/36
3/36
2/36
1/36
For example, there are four outcomes which make X equal to 5.
9
SMU Classification: Restricted
PROBABILITY DISTRIBUTION EXAMPLE: X IS THE SUM OF TWO DICE
red
green
1
2
3
4
5
6
1
2
3
4
5
6
7
2
3
4
5
6
7
8
3
4
5
6
7
8
9
4
5
6
7
8
9
10
5
6
7
8
9
10
11
6
7
8
9
10
11
12
X
f
p
2
3
4
5
6
7
8
9
10
11
12
1
2
3
4
5
6
5
4
3
2
1
1/36
2/36
3/36
4/36
5/36
6/36
5/36
4/36
3/36
2/36
1/36
Similarly you can work out the frequencies for all the other values of X.
10
SMU Classification: Restricted
PROBABILITY DISTRIBUTION EXAMPLE: X IS THE SUM OF TWO DICE
red
green
1
2
3
4
5
6
1
2
3
4
5
6
7
2
3
4
5
6
7
8
3
4
5
6
7
8
9
4
5
6
7
8
9
10
5
6
7
8
9
10
11
6
7
8
9
10
11
12
X
f
p
2
3
4
5
6
7
8
9
10
11
12
1
2
3
4
5
6
5
4
3
2
1
1/36
2/36
3/36
4/36
5/36
6/36
5/36
4/36
3/36
2/36
1/36
Finally we will derive the probability of obtaining each value of X.
11
SMU Classification: Restricted
PROBABILITY DISTRIBUTION EXAMPLE: X IS THE SUM OF TWO DICE
red
green
1
2
3
4
5
6
1
2
3
4
5
6
7
2
3
4
5
6
7
8
3
4
5
6
7
8
9
4
5
6
7
8
9
10
5
6
7
8
9
10
11
6
7
8
9
10
11
12
X
f
p
2
3
4
5
6
7
8
9
10
11
12
1
2
3
4
5
6
5
4
3
2
1
1/36
2/36
3/36
4/36
5/36
6/36
5/36
4/36
3/36
2/36
1/36
If there is 1/6 probability of obtaining each number on the red die, and the same on the
green die, each outcome in the table will occur with 1/36 probability.
12
SMU Classification: Restricted
PROBABILITY DISTRIBUTION EXAMPLE: X IS THE SUM OF TWO DICE
red
green
1
2
3
4
5
6
1
2
3
4
5
6
7
2
3
4
5
6
7
8
3
4
5
6
7
8
9
4
5
6
7
8
9
10
5
6
7
8
9
10
11
6
7
8
9
10
11
12
X
f
p
2
3
4
5
6
7
8
9
10
11
12
1
2
3
4
5
6
5
4
3
2
1
1/36
2/36
3/36
4/36
5/36
6/36
5/36
4/36
3/36
2/36
1/36
Hence to obtain the probabilities associated with the different values of X, we divide the
frequencies by 36.
13
SMU Classification: Restricted
PROBABILITY DISTRIBUTION EXAMPLE: X IS THE SUM OF TWO DICE
probability
1
36
2
2
__
36
3
__
36
4
__
36
5
__
36
6
__
36
5
__
36
4
__
36
3
__
36
2
__
36
3
4
5
6
7
8
9 10 11 12
1
36
X
The distribution is shown graphically. in this example it is symmetrical, highest for X equal
to 7 and declining on either side.
14
SMU Classification: Restricted
Introduction to Econometrics
Review of Probability
2. Expectation
SMU Classification: Restricted
EXPECTED VALUE OF A RANDOM VARIABLE
Definition of E(X), the expected value of X:
n
E ( X )  x1 p1  ...  xn pn   xi pi
i 1
The expected value of a random variable, also known as its population mean, is the
weighted average of its possible values, the weights being the probabilities attached to the
values.
1
SMU Classification: Restricted
EXPECTED VALUE OF A RANDOM VARIABLE
Definition of E(X), the expected value of X:
n
E ( X )  x1 p1  ...  xn pn   xi pi
i 1
Note that the sum of the probabilities must be unity, so there is no need to divide by the
sum of the weights.
2
SMU Classification: Restricted
EXPECTED VALUE OF A RANDOM VARIABLE
xi
x1
x2
x3
x4
x5
x6
x7
x8
x9
x10
x11
pi
p1
p2
p3
p4
p5
p6
p7
p8
p9
p10
p11
xi pi
x1 p1
x2 p2
x3 p3
x4 p4
x5 p5
x6 p6
x7 p7
x8 p8
x9 p9
x10 p10
x11 p11
S xi pi = E(X)
xi
pi
xi pi
2
1/36
2/36
3
2/36
6/36
4
3/36
12/36
5
4/36
20/36
6
5/36
30/36
7
6/36
42/36
8
5/36
40/36
9
4/36
36/36
10
3/36
30/36
11
2/36
22/36
12
1/36
12/36
This sequence shows how the expected value is calculated, first in abstract and then with
the random variable defined in the first sequence. We begin by listing the possible values
of X.
3
SMU Classification: Restricted
EXPECTED VALUE OF A RANDOM VARIABLE
xi
x1
x2
x3
x4
x5
x6
x7
x8
x9
x10
x11
pi
p1
p2
p3
p4
p5
p6
p7
p8
p9
p10
p11
xi pi
x1 p1
x2 p2
x3 p3
x4 p4
x5 p5
x6 p6
x7 p7
x8 p8
x9 p9
x10 p10
x11 p11
S xi pi = E(X)
xi
pi
xi pi
2
1/36
2/36
3
2/36
6/36
4
3/36
12/36
5
4/36
20/36
6
5/36
30/36
7
6/36
42/36
8
5/36
40/36
9
4/36
36/36
10
3/36
30/36
11
2/36
22/36
12
1/36
12/36
Next we list the probabilities attached to the different possible values of X.
4
SMU Classification: Restricted
EXPECTED VALUE OF A RANDOM VARIABLE
xi
x1
x2
x3
x4
x5
x6
x7
x8
x9
x10
x11
pi
p1
p2
p3
p4
p5
p6
p7
p8
p9
p10
p11
xi pi
x1 p1
x2 p2
x3 p3
x4 p4
x5 p5
x6 p6
x7 p7
x8 p8
x9 p9
x10 p10
x11 p11
S xi pi = E(X)
xi
pi
xi pi
2
1/36
2/36
3
2/36
6/36
4
3/36
12/36
5
4/36
20/36
6
5/36
30/36
7
6/36
42/36
8
5/36
40/36
9
4/36
36/36
10
3/36
30/36
11
2/36
22/36
12
1/36
12/36
Then we define a column in which the values are weighted by the corresponding
probabilities.
5
SMU Classification: Restricted
EXPECTED VALUE OF A RANDOM VARIABLE
xi
x1
x2
x3
x4
x5
x6
x7
x8
x9
x10
x11
pi
p1
p2
p3
p4
p5
p6
p7
p8
p9
p10
p11
xi pi
x1 p1
x2 p2
x3 p3
x4 p4
x5 p5
x6 p6
x7 p7
x8 p8
x9 p9
x10 p10
x11 p11
S xi pi = E(X)
xi
pi
xi pi
2
1/36
2/36
3
2/36
6/36
4
3/36
12/36
5
4/36
20/36
6
5/36
30/36
7
6/36
42/36
8
5/36
40/36
9
4/36
36/36
10
3/36
30/36
11
2/36
22/36
12
1/36
12/36
We do this for each value separately.
6
SMU Classification: Restricted
EXPECTED VALUE OF A RANDOM VARIABLE
xi
x1
x2
x3
x4
x5
x6
x7
x8
x9
x10
x11
pi
p1
p2
p3
p4
p5
p6
p7
p8
p9
p10
p11
xi pi
x1 p1
x2 p2
x3 p3
x4 p4
x5 p5
x6 p6
x7 p7
x8 p8
x9 p9
x10 p10
x11 p11
S xi pi = E(X)
xi
pi
xi pi
2
1/36
2/36
3
2/36
6/36
4
3/36
12/36
5
4/36
20/36
6
5/36
30/36
7
6/36
42/36
8
5/36
40/36
9
4/36
36/36
10
3/36
30/36
11
2/36
22/36
12
1/36
12/36
Here we are assuming that n, the number of possible values, is equal to 11, but it could be
any number.
7
SMU Classification: Restricted
EXPECTED VALUE OF A RANDOM VARIABLE
xi
x1
x2
x3
x4
x5
x6
x7
x8
x9
x10
x11
pi
p1
p2
p3
p4
p5
p6
p7
p8
p9
p10
p11
xi pi
x1 p1
x2 p2
x3 p3
x4 p4
x5 p5
x6 p6
x7 p7
x8 p8
x9 p9
x10 p10
x11 p11
S xi pi = E(X)
xi
pi
xi pi
2
1/36
2/36
3
2/36
6/36
4
3/36
12/36
5
4/36
20/36
6
5/36
30/36
7
6/36
42/36
8
5/36
40/36
9
4/36
36/36
10
3/36
30/36
11
2/36
22/36
12
1/36
12/36
The expected value is the sum of the entries in the third column.
8
SMU Classification: Restricted
EXPECTED VALUE OF A RANDOM VARIABLE
xi
x1
x2
x3
x4
x5
x6
x7
x8
x9
x10
x11
pi
p1
p2
p3
p4
p5
p6
p7
p8
p9
p10
p11
xi pi
x1 p1
x2 p2
x3 p3
x4 p4
x5 p5
x6 p6
x7 p7
x8 p8
x9 p9
x10 p10
x11 p11
S xi pi = E(X)
xi
pi
xi pi
2
1/36
2/36
3
2/36
6/36
4
3/36
12/36
5
4/36
20/36
6
5/36
30/36
7
6/36
42/36
8
5/36
40/36
9
4/36
36/36
10
3/36
30/36
11
2/36
22/36
12
1/36
12/36
The random variable X defined in the previous sequence could be any of the integers from 2
to 12 with probabilities as shown.
9
SMU Classification: Restricted
EXPECTED VALUE OF A RANDOM VARIABLE
xi
x1
x2
x3
x4
x5
x6
x7
x8
x9
x10
x11
pi
p1
p2
p3
p4
p5
p6
p7
p8
p9
p10
p11
xi pi
x1 p1
x2 p2
x3 p3
x4 p4
x5 p5
x6 p6
x7 p7
x8 p8
x9 p9
x10 p10
x11 p11
S xi pi = E(X)
xi
pi
xi pi
2
1/36
2/36
3
2/36
6/36
4
3/36
12/36
5
4/36
20/36
6
5/36
30/36
7
6/36
42/36
8
5/36
40/36
9
4/36
36/36
10
3/36
30/36
11
2/36
22/36
12
1/36
12/36
X could be equal to 2 with probability 1/36, so the first entry in the calculation of the
expected value is 2/36.
10
SMU Classification: Restricted
EXPECTED VALUE OF A RANDOM VARIABLE
xi
x1
x2
x3
x4
x5
x6
x7
x8
x9
x10
x11
pi
p1
p2
p3
p4
p5
p6
p7
p8
p9
p10
p11
xi pi
x1 p1
x2 p2
x3 p3
x4 p4
x5 p5
x6 p6
x7 p7
x8 p8
x9 p9
x10 p10
x11 p11
S xi pi = E(X)
xi
pi
xi pi
2
1/36
2/36
3
2/36
6/36
4
3/36
12/36
5
4/36
20/36
6
5/36
30/36
7
6/36
42/36
8
5/36
40/36
9
4/36
36/36
10
3/36
30/36
11
2/36
22/36
12
1/36
12/36
The probability of x being equal to 3 was 2/36, so the second entry is 6/36.
11
SMU Classification: Restricted
EXPECTED VALUE OF A RANDOM VARIABLE
xi
x1
x2
x3
x4
x5
x6
x7
x8
x9
x10
x11
pi
p1
p2
p3
p4
p5
p6
p7
p8
p9
p10
p11
xi pi
x1 p1
x2 p2
x3 p3
x4 p4
x5 p5
x6 p6
x7 p7
x8 p8
x9 p9
x10 p10
x11 p11
S xi pi = E(X)
xi
pi
xi pi
2
1/36
2/36
3
2/36
6/36
4
3/36
12/36
5
4/36
20/36
6
5/36
30/36
7
6/36
42/36
8
5/36
40/36
9
4/36
36/36
10
3/36
30/36
11
2/36
22/36
12
1/36
12/36
Similarly for the other 9 possible values.
12
SMU Classification: Restricted
EXPECTED VALUE OF A RANDOM VARIABLE
xi
x1
x2
x3
x4
x5
x6
x7
x8
x9
x10
x11
pi
p1
p2
p3
p4
p5
p6
p7
p8
p9
p10
p11
xi pi
x1 p1
x2 p2
x3 p3
x4 p4
x5 p5
x6 p6
x7 p7
x8 p8
x9 p9
x10 p10
x11 p11
S xi pi = E(X)
xi
pi
xi pi
2
1/36
2/36
3
2/36
6/36
4
3/36
12/36
5
4/36
20/36
6
5/36
30/36
7
6/36
42/36
8
5/36
40/36
9
4/36
36/36
10
3/36
30/36
11
2/36
22/36
12
1/36
12/36
252/36
To obtain the expected value, we sum the entries in this column.
13
SMU Classification: Restricted
EXPECTED VALUE OF A RANDOM VARIABLE
xi
x1
x2
x3
x4
x5
x6
x7
x8
x9
x10
x11
pi
p1
p2
p3
p4
p5
p6
p7
p8
p9
p10
p11
xi pi
x1 p1
x2 p2
x3 p3
x4 p4
x5 p5
x6 p6
x7 p7
x8 p8
x9 p9
x10 p10
x11 p11
S xi pi = E(X)
xi
pi
xi pi
2
1/36
2/36
3
2/36
6/36
4
3/36
12/36
5
4/36
20/36
6
5/36
30/36
7
6/36
42/36
8
5/36
40/36
9
4/36
36/36
10
3/36
30/36
11
2/36
22/36
12
1/36
12/36
252/36 = 7
The expected value turns out to be 7. Actually, this was obvious anyway. We saw in the
previous sequence that the distribution is symmetrical about 7.
14
SMU Classification: Restricted
EXPECTED VALUE OF A RANDOM VARIABLE
Alternative notation for E(X):
n
E ( X )  x1 p1  ...  x n pn   x i pi  m X
i 1
Very often the expected value of a random variable is represented by m, the Greek m. If
there is more than one random variable, their expected values are differentiated by adding
subscripts to m.
15
Introduction to Econometrics
Review of Probability
3. Expectation of a Function of a
Random Variable
SMU Classification: Restricted
EXPECTED VALUE OF A FUNCTION OF A RANDOM VARIABLE
Definition of
E  g  X  , the expected value of a function of X:
n
E  g  X   g  x1  p1  ...  g  x n  pn   g  x i  pi
i 1
To find the expected value of a function of a random variable, one calculates all the possible
values of the function, weights them by the corresponding probabilities, and sums the
results.
1
SMU Classification: Restricted
EXPECTED VALUE OF A FUNCTION OF A RANDOM VARIABLE
Definition of
E  g  X  , the expected value of a function of X:
n
E  g  X   g  x1  p1  ...  g  x n  pn   g  x i  pi
i 1
Example:
E X
2
 x
2
1
n
p1  ...  x pn   x i2 pi
2
n
i 1
For example, the expected value of X2 is found by calculating all its possible values,
multiplying them by the corresponding probabilities, and summing.
2
SMU Classification: Restricted
EXPECTED VALUE OF A FUNCTION OF A RANDOM VARIABLE
xi
x1
x2
x3
…
…
…
…
…
…
…
xn
pi
p1
p2
p3
…
…
…
…
…
…
…
pn
g(xi)
g(x1)
g(x2)
g(x3)
…...
…...
…...
…...
…...
…...
…...
g(xn)
g(xi ) pi
g(x1) p1
g(x2) p2
g(x3) p3
……...
……...
……...
……...
……...
……...
……...
g(xn) pn
S g(xi) pi
xi
pi
xi2
xi2 pi
2
1/36
4
0.11
3
2/36
9
0.50
4
3/36
16
1.33
5
4/36
25
2.78
6
5/36
36
5.00
7
6/36
49
8.17
8
5/36
64
8.89
9
4/36
81
9.00
10
3/36
100
8.83
11
2/36
121
6.72
12
1/36
144
4.00
54.83
The calculation of the expected value of a function of a random variable will be outlined in
general and then illustrated with an example.
3
SMU Classification: Restricted
EXPECTED VALUE OF A FUNCTION OF A RANDOM VARIABLE
xi
x1
x2
x3
…
…
…
…
…
…
…
xn
pi
p1
p2
p3
…
…
…
…
…
…
…
pn
g(xi)
g(x1)
g(x2)
g(x3)
…...
…...
…...
…...
…...
…...
…...
g(xn)
g(xi ) pi
g(x1) p1
g(x2) p2
g(x3) p3
……...
……...
……...
……...
……...
……...
……...
g(xn) pn
S g(xi) pi
xi
pi
xi2
xi2 pi
2
1/36
4
0.11
3
2/36
9
0.50
4
3/36
16
1.33
5
4/36
25
2.78
6
5/36
36
5.00
7
6/36
49
8.17
8
5/36
64
8.89
9
4/36
81
9.00
10
3/36
100
8.83
11
2/36
121
6.72
12
1/36
144
4.00
54.83
First one makes a list of the possible values of X and the corresponding probabilities.
4
SMU Classification: Restricted
EXPECTED VALUE OF A FUNCTION OF A RANDOM VARIABLE
xi
x1
x2
x3
…
…
…
…
…
…
…
xn
pi
p1
p2
p3
…
…
…
…
…
…
…
pn
g(xi)
g(x1)
g(x2)
g(x3)
…...
…...
…...
…...
…...
…...
…...
g(xn)
g(xi ) pi
g(x1) p1
g(x2) p2
g(x3) p3
……...
……...
……...
……...
……...
……...
……...
g(xn) pn
xi
pi
xi2
xi2 pi
2
1/36
4
0.11
3
2/36
9
0.50
4
3/36
16
1.33
5
4/36
25
2.78
6
5/36
36
5.00
7
6/36
49
8.17
8
5/36
64
8.89
9
4/36
81
9.00
10
3/36
100
8.83
11
2/36
121
6.72
12
1/36
144
4.00
S g(xi) pi
54.83
Next the function of X is calculated for each possible value of X.
5
SMU Classification: Restricted
EXPECTED VALUE OF A FUNCTION OF A RANDOM VARIABLE
xi
x1
x2
x3
…
…
…
…
…
…
…
xn
pi
p1
p2
p3
…
…
…
…
…
…
…
pn
g(xi)
g(x1)
g(x2)
g(x3)
…...
…...
…...
…...
…...
…...
…...
g(xn)
g(xi ) pi
g(x1) p1
g(x2) p2
g(x3) p3
……...
……...
……...
……...
……...
……...
……...
g(xn) pn
S g(xi) pi
xi
pi
xi2
xi2 pi
2
1/36
4
0.11
3
2/36
9
0.50
4
3/36
16
1.33
5
4/36
25
2.78
6
5/36
36
5.00
7
6/36
49
8.17
8
5/36
64
8.89
9
4/36
81
9.00
10
3/36
100
8.83
11
2/36
121
6.72
12
1/36
144
4.00
54.83
Then, one at a time, the value of the function is weighted by its corresponding probability.
6
SMU Classification: Restricted
EXPECTED VALUE OF A FUNCTION OF A RANDOM VARIABLE
xi
x1
x2
x3
…
…
…
…
…
…
…
xn
pi
p1
p2
p3
…
…
…
…
…
…
…
pn
g(xi)
g(x1)
g(x2)
g(x3)
…...
…...
…...
…...
…...
…...
…...
g(xn)
g(xi ) pi
g(x1) p1
g(x2) p2
g(x3) p3
……...
……...
……...
……...
……...
……...
……...
g(xn) pn
xi
pi
xi2
xi2 pi
2
1/36
4
0.11
3
2/36
9
0.50
4
3/36
16
1.33
5
4/36
25
2.78
6
5/36
36
5.00
7
6/36
49
8.17
8
5/36
64
8.89
9
4/36
81
9.00
10
3/36
100
8.83
11
2/36
121
6.72
12
1/36
144
4.00
S g(xi) pi
54.83
This is done individually for each possible value of X.
7
SMU Classification: Restricted
EXPECTED VALUE OF A FUNCTION OF A RANDOM VARIABLE
xi
x1
x2
x3
…
…
…
…
…
…
…
xn
pi
p1
p2
p3
…
…
…
…
…
…
…
pn
g(xi)
g(x1)
g(x2)
g(x3)
…...
…...
…...
…...
…...
…...
…...
g(xn)
g(xi ) pi
g(x1) p1
g(x2) p2
g(x3) p3
……...
……...
……...
……...
……...
……...
……...
g(xn) pn
xi
pi
xi2
xi2 pi
2
1/36
4
0.11
3
2/36
9
0.50
4
3/36
16
1.33
5
4/36
25
2.78
6
5/36
36
5.00
7
6/36
49
8.17
8
5/36
64
8.89
9
4/36
81
9.00
10
3/36
100
8.83
11
2/36
121
6.72
12
1/36
144
4.00
S g(xi) pi
54.83
The sum of the weighted values is the expected value of the function of X.
8
SMU Classification: Restricted
EXPECTED VALUE OF A FUNCTION OF A RANDOM VARIABLE
xi
x1
x2
x3
…
…
…
…
…
…
…
xn
pi
p1
p2
p3
…
…
…
…
…
…
…
pn
g(xi)
g(x1)
g(x2)
g(x3)
…...
…...
…...
…...
…...
…...
…...
g(xn)
g(xi ) pi
g(x1) p1
g(x2) p2
g(x3) p3
……...
……...
……...
……...
……...
……...
……...
g(xn) pn
S g(xi) pi
xi
pi
xi2
xi2 pi
2
1/36
4
0.11
3
2/36
9
0.50
4
3/36
16
1.33
5
4/36
25
2.78
6
5/36
36
5.00
7
6/36
49
8.17
8
5/36
64
8.89
9
4/36
81
9.00
10
3/36
100
8.83
11
2/36
121
6.72
12
1/36
144
4.00
54.83
The process will be illustrated for X2, where X is the random variable defined in the first
sequence. The 11 possible values of X and the corresponding probabilities are listed.
9
SMU Classification: Restricted
EXPECTED VALUE OF A FUNCTION OF A RANDOM VARIABLE
xi
x1
x2
x3
…
…
…
…
…
…
…
xn
pi
p1
p2
p3
…
…
…
…
…
…
…
pn
g(xi)
g(x1)
g(x2)
g(x3)
…...
…...
…...
…...
…...
…...
…...
g(xn)
g(xi ) pi
g(x1) p1
g(x2) p2
g(x3) p3
……...
……...
……...
……...
……...
……...
……...
g(xn) pn
S g(xi) pi
xi
pi
xi2
xi2 pi
2
1/36
4
0.11
3
2/36
9
0.50
4
3/36
16
1.33
5
4/36
25
2.78
6
5/36
36
5.00
7
6/36
49
8.17
8
5/36
64
8.89
9
4/36
81
9.00
10
3/36
100
8.83
11
2/36
121
6.72
12
1/36
144
4.00
54.83
First one calculates the possible values of X2.
10
SMU Classification: Restricted
EXPECTED VALUE OF A FUNCTION OF A RANDOM VARIABLE
xi
x1
x2
x3
…
…
…
…
…
…
…
xn
pi
p1
p2
p3
…
…
…
…
…
…
…
pn
g(xi)
g(x1)
g(x2)
g(x3)
…...
…...
…...
…...
…...
…...
…...
g(xn)
g(xi ) pi
g(x1) p1
g(x2) p2
g(x3) p3
……...
……...
……...
……...
……...
……...
……...
g(xn) pn
S g(xi) pi
xi
pi
xi2
xi2 pi
2
1/36
4
0.11
3
2/36
9
0.50
4
3/36
16
1.33
5
4/36
25
2.78
6
5/36
36
5.00
7
6/36
49
8.17
8
5/36
64
8.89
9
4/36
81
9.00
10
3/36
100
8.83
11
2/36
121
6.72
12
1/36
144
4.00
54.83
The first value is 4, which arises when X is equal to 2. The probability of X being equal to 2
is 1/36, so the weighted function is 4/36, which we shall write in decimal form as 0.11.
11
SMU Classification: Restricted
EXPECTED VALUE OF A FUNCTION OF A RANDOM VARIABLE
xi
x1
x2
x3
…
…
…
…
…
…
…
xn
pi
p1
p2
p3
…
…
…
…
…
…
…
pn
g(xi)
g(x1)
g(x2)
g(x3)
…...
…...
…...
…...
…...
…...
…...
g(xn)
g(xi ) pi
g(x1) p1
g(x2) p2
g(x3) p3
……...
……...
……...
……...
……...
……...
……...
g(xn) pn
S g(xi) pi
xi
pi
xi2
xi2 pi
2
1/36
4
0.11
3
2/36
9
0.50
4
3/36
16
1.33
5
4/36
25
2.78
6
5/36
36
5.00
7
6/36
49
8.17
8
5/36
64
8.89
9
4/36
81
9.00
10
3/36
100
8.83
11
2/36
121
6.72
12
1/36
144
4.00
54.83
Similarly for all the other possible values of X.
12
SMU Classification: Restricted
EXPECTED VALUE OF A FUNCTION OF A RANDOM VARIABLE
xi
x1
x2
x3
…
…
…
…
…
…
…
xn
pi
p1
p2
p3
…
…
…
…
…
…
…
pn
g(xi)
g(x1)
g(x2)
g(x3)
…...
…...
…...
…...
…...
…...
…...
g(xn)
g(xi ) pi
g(x1) p1
g(x2) p2
g(x3) p3
……...
……...
……...
……...
……...
……...
……...
g(xn) pn
S g(xi) pi
xi
pi
xi2
xi2 pi
2
1/36
4
0.11
3
2/36
9
0.50
4
3/36
16
1.33
5
4/36
25
2.78
6
5/36
36
5.00
7
6/36
49
8.17
8
5/36
64
8.89
9
4/36
81
9.00
10
3/36
100
8.83
11
2/36
121
6.72
12
1/36
144
4.00
54.83
The expected value of X2 is the sum of its weighted values in the final column. It is equal to
54.83. It is the average value of the figures in the previous column, taking the differing
probabilities into account.
13
SMU Classification: Restricted
EXPECTED VALUE OF A FUNCTION OF A RANDOM VARIABLE
xi
x1
x2
x3
…
…
…
…
…
…
…
xn
pi
g(xi) g(xi ) pi
p1
g(x1) g(x1) p1
p2
g(x2) g(x2) p2
p3
g(x3) g(x3) p3
… …... ……...
… …... ……...
E… X 2 …...
 54.83……...
…
……...
E  X …...
7
… …... ……...
2
E… X 2 …...
  E  X……...
… …... ……...
pn
g(xn) g(xn) pn
S g(xi) pi
xi
pi
xi2
xi2 pi
2
1/36
4
0.11
3
2/36
9
0.50
4
3/36
16
1.33
5
4/36
25
2.78
6
5/36
36
5.00
7
6/36
49
8.17
8
5/36
64
8.89
9
4/36
81
9.00
10
3/36
100
8.83
11
2/36
121
6.72
12
1/36
144
4.00
54.83
Note that E(X2) is not the same thing as E(X), squared. In the previous sequence we saw
that E(X) for this example was 7. Its square is 49.
14
Introduction to Econometrics
Review of Probability
4. Expected Value Rules
SMU Classification: Restricted
EXPECTED VALUE RULES
1.
E  X  Y  Z   E  X   E Y   E  Z 
This sequence states the rules for manipulating expected values. First, the additive rule.
The expected value of the sum of two random variables is the sum of their expected values.
1
SMU Classification: Restricted
EXPECTED VALUE RULES
1.
E  X  Y  Z   E  X   E Y   E  Z 
Here the sum consists of three variables. But the rule generalizes to any number.
2
SMU Classification: Restricted
EXPECTED VALUE RULES
1.
2.
E  X  Y  Z   E  X   E Y   E  Z 
E bX   bE  X 
The second rule is the multiplicative rule. The expected value of a variable that has been
multiplied by a constant) is equal to the constant multiplied by the expected value of the
variable.
3
SMU Classification: Restricted
EXPECTED VALUE RULES
1.
E  X  Y  Z   E  X   E Y   E  Z 
2.
E bX   bE  X 
Example:
E 3 X   3 E  X 
For example, the expected value of 3X is three times the expected value of X.
4
SMU Classification: Restricted
EXPECTED VALUE RULES
1.
2.
3.
E  X  Y  Z   E  X   E Y   E  Z 
E bX   bE  X 
E b   b
Finally, the expected value of a constant is just the constant. Of course this is obvious.
5
SMU Classification: Restricted
EXPECTED VALUE RULES
1.
2.
3.
E  X  Y  Z   E  X   E Y   E  Z 
E bX   bE  X 
E b   b
Y  b1  b2 X
E Y   E b1  b2 X 
 E b1   E b2 X 
 b1  b2 E  X 
As an exercise, we will use the rules to simplify the expected value of an expression.
Suppose that we are interested in the expected value of a variable Y, where Y = b1 + b2X.
6
SMU Classification: Restricted
EXPECTED VALUE RULES
1.
2.
3.
E  X  Y  Z   E  X   E Y   E  Z 
E bX   bE  X 
E b   b
Y  b1  b2 X
E Y   E b1  b2 X 
 E b1   E b2 X 
 b1  b2 E  X 
We use the first rule to break up the expected value into its two components.
7
SMU Classification: Restricted
EXPECTED VALUE RULES
1.
2.
3.
E  X  Y  Z   E  X   E Y   E  Z 
E bX   bE  X 
E b   b
Y  b1  b2 X
E Y   E b1  b2 X 
 E b1   E b2 X 
 b1  b2 E  X 
Then we use the second rule to replace E(b2X) by b2E(X) and the third rule to simplify E(b1)
to just b1. This is as far as we can go in this example.
8
Introduction to Econometrics
Review of Probability
5. Variance
SMU Classification: Restricted
POPULATION VARIANCE OF A DISCRETE RANDOM VARIABLE
Population variance of X:

E X  m
2
  x

E X  m
2

1  m  p1  ...   x n  m  pn
n
2
2
   x i  m  pi
2
i 1
Clearly the population variance of X is defined as the expected value of a function of a
random variable X. Where the function of interest to us is the squared deviation from the
population mean.
1
SMU Classification: Restricted
POPULATION VARIANCE OF A DISCRETE RANDOM VARIABLE
Population variance of X:

E X  m
2
  x

E X  m
2

1  m  p1  ...   x n  m  pn
n
2
2
   x i  m  pi
2
i 1
The expected value of the squared deviation is known as the population variance of X. It is
a measure of the dispersion of the distribution of X about its population mean.
2
SMU Classification: Restricted
POPULATION VARIANCE OF A DISCRETE RANDOM VARIABLE
xi
pi
xi – m
(xi – m)2
(xi – m)2 pi
2
3
4
1/36
2/36
3/36
–5
–4
–3
25
16
9
0.69
0.89
0.75
5
6
7
8
4/36
5/36
6/36
5/36
–2
–1
0
1
4
1
0
1
0.44
0.14
0.00
0.14
9
10
11
12
4/36
3/36
2/36
1/36
2
3
4
5
4
9
16
25
0.44
0.75
0.89
0.69
5.83
We will calculate the population variance of the random variable X defined in the first
sequence. We start as usual by listing the possible values of X and the corresponding
probabilities.
3
SMU Classification: Restricted
POPULATION VARIANCE OF A DISCRETE RANDOM VARIABLE
xi
pi
xi – m
(xi – m)2
2
3
4
1/36
2/36
3/36
–5
–4
–3
25
16
9
5
6
7
8
4/36
5/36
6/36
5/36
–2
–1
0
1
9
10
11
12
4/36
3/36
2/36
1/36
2
3
4
5
4
1
0m X
1
4
9
16
25
(xi – m)2 pi
0.69
0.89
0.75
 E
0.44
0.14
X0.00
7
0.14

0.44
0.75
0.89
0.69
5.83
Next we need a column giving the deviations of the possible values of X about its
population mean. In the second sequence we saw that the population mean of X was 7.
4
SMU Classification: Restricted
POPULATION VARIANCE OF A DISCRETE RANDOM VARIABLE
xi
pi
xi – m
(xi – m)2
2
3
4
1/36
2/36
3/36
–5
–4
–3
25
16
9
5
6
7
8
4/36
5/36
6/36
5/36
–2
–1
0
1
9
10
11
12
4/36
3/36
2/36
1/36
2
3
4
5
4
1
0m X
1
4
9
16
25
(xi – m)2 pi
0.69
0.89
0.75
 E
0.44
0.14
X0.00
7
0.14

0.44
0.75
0.89
0.69
5.83
When X is equal to 2, the deviation is –5.
5
SMU Classification: Restricted
POPULATION VARIANCE OF A DISCRETE RANDOM VARIABLE
xi
pi
xi – m
(xi – m)2
2
3
4
1/36
2/36
3/36
–5
–4
–3
25
16
9
5
6
7
8
4/36
5/36
6/36
5/36
–2
–1
0
1
9
10
11
12
4/36
3/36
2/36
1/36
2
3
4
5
4
1
0m X
1
4
9
16
25
(xi – m)2 pi
0.69
0.89
0.75
 E
0.44
0.14
X0.00
7
0.14

0.44
0.75
0.89
0.69
5.83
Similarly for all the other possible values.
6
SMU Classification: Restricted
POPULATION VARIANCE OF A DISCRETE RANDOM VARIABLE
xi
pi
xi – m
(xi – m)2
(xi – m)2 pi
2
3
4
1/36
2/36
3/36
–5
–4
–3
25
16
9
0.69
0.89
0.75
5
6
7
8
4/36
5/36
6/36
5/36
–2
–1
0
1
4
1
0
1
0.44
0.14
0.00
0.14
9
10
11
12
4/36
3/36
2/36
1/36
2
3
4
5
4
9
16
25
0.44
0.75
0.89
0.69
5.83
Next we need a column giving the squared deviations. When X is equal to 2, the squared
deviation is 25.
7
SMU Classification: Restricted
POPULATION VARIANCE OF A DISCRETE RANDOM VARIABLE
xi
pi
xi – m
(xi – m)2
(xi – m)2 pi
2
3
4
1/36
2/36
3/36
–5
–4
–3
25
16
9
0.69
0.89
0.75
5
6
7
8
4/36
5/36
6/36
5/36
–2
–1
0
1
4
1
0
1
0.44
0.14
0.00
0.14
9
10
11
12
4/36
3/36
2/36
1/36
2
3
4
5
4
9
16
25
0.44
0.75
0.89
0.69
5.83
Similarly for the other values of X.
8
SMU Classification: Restricted
POPULATION VARIANCE OF A DISCRETE RANDOM VARIABLE
xi
pi
xi – m
(xi – m)2
(xi – m)2 pi
2
3
4
1/36
2/36
3/36
–5
–4
–3
25
16
9
0.69
0.89
0.75
5
6
7
8
4/36
5/36
6/36
5/36
–2
–1
0
1
4
1
0
1
0.44
0.14
0.00
0.14
9
10
11
12
4/36
3/36
2/36
1/36
2
3
4
5
4
9
16
25
0.44
0.75
0.89
0.69
5.83
Now we start weighting the squared deviations by the corresponding probabilities. What do
you think the weighted average will be? Have a guess.
9
SMU Classification: Restricted
POPULATION VARIANCE OF A DISCRETE RANDOM VARIABLE
xi
pi
xi – m
(xi – m)2
(xi – m)2 pi
2
3
4
1/36
2/36
3/36
–5
–4
–3
25
16
9
0.69
0.89
0.75
5
6
7
8
4/36
5/36
6/36
5/36
–2
–1
0
1
4
1
0
1
0.44
0.14
0.00
0.14
9
10
11
12
4/36
3/36
2/36
1/36
2
3
4
5
4
9
16
25
0.44
0.75
0.89
0.69
5.83
A reason for making an initial guess is that it may help you to identify an arithmetical error,
if you make one. If the initial guess and the outcome are very different, that is a warning.
10
SMU Classification: Restricted
POPULATION VARIANCE OF A DISCRETE RANDOM VARIABLE
xi
pi
xi – m
(xi – m)2
(xi – m)2 pi
2
3
4
1/36
2/36
3/36
–5
–4
–3
25
16
9
0.69
0.89
0.75
5
6
7
8
4/36
5/36
6/36
5/36
–2
–1
0
1
4
1
0
1
0.44
0.14
0.00
0.14
9
10
11
12
4/36
3/36
2/36
1/36
2
3
4
5
4
9
16
25
0.44
0.75
0.89
0.69
5.83
We calculate all the weighted squared deviations.
11
SMU Classification: Restricted
POPULATION VARIANCE OF A DISCRETE RANDOM VARIABLE
xi
pi
xi – m
(xi – m)2
(xi – m)2 pi
2
3
4
1/36
2/36
3/36
–5
–4
–3
25
16
9
0.69
0.89
0.75
5
6
7
8
4/36
5/36
6/36
5/36
–2
–1
0
1
4
1
0
1
0.44
0.14
0.00
0.14
9
10
11
12
4/36
3/36
2/36
1/36
2
3
4
5
4
9
16
25
0.44
0.75
0.89
0.69
5.83
The sum is the population variance of X.
12
SMU Classification: Restricted
POPULATION VARIANCE OF A DISCRETE RANDOM VARIABLE
Population variance of X

E X  m
2

s X2
In equations, the population variance of X is usually written sX2, s being the Greek s.
13
SMU Classification: Restricted
POPULATION VARIANCE OF A DISCRETE RANDOM VARIABLE
Standard deviation of X

E X  m
2

sX
The standard deviation of X is the square root of its population variance. Usually written sx,
it is an alternative measure of dispersion. It has the same units as X.
14
Introduction to Econometrics
Review of Probability
6. Variance Formula
SMU Classification: Restricted
ALTERNATIVE EXPRESSION FOR POPULATION VARIANCE
s X2  E  X 2   m 2
This sequence derives an alternative expression for the population variance of a random
variable. It provides an opportunity for practising the use of the expected value rules.
1
SMU Classification: Restricted
ALTERNATIVE EXPRESSION FOR POPULATION VARIANCE
s X2  E  X 2   m 2

s X2  E  X  m 
2

 E  X 2  2 mX  m 2 
 E  X 2   E  2mX   E m 2 
 E  X 2   2 mE  X   m 2
 E  X 2   2m 2  m 2
 E X 2   m 2
We start with the definition of the population variance of X.
2
SMU Classification: Restricted
ALTERNATIVE EXPRESSION FOR POPULATION VARIANCE
s X2  E  X 2   m 2

s X2  E  X  m 
2

 E  X 2  2 mX  m 2 
 E  X 2   E  2mX   E m 2 
 E  X 2   2 mE  X   m 2
 E  X 2   2m 2  m 2
 E X 2   m 2
We expand the quadratic.
3
SMU Classification: Restricted
ALTERNATIVE EXPRESSION FOR POPULATION VARIANCE
s X2  E  X 2   m 2

s X2  E  X  m 
2

 E  X 2  2 mX  m 2 
 E  X 2   E  2mX   E m 2 
 E  X 2   2 mE  X   m 2
 E  X 2   2m 2  m 2
 E X 2   m 2
Now the first expected value rule is used to decompose the expression into three separate
expected values.
4
SMU Classification: Restricted
ALTERNATIVE EXPRESSION FOR POPULATION VARIANCE
s X2  E  X 2   m 2

s X2  E  X  m 
2

 E  X 2  2 mX  m 2 
 E  X 2   E  2mX   E m 2 
 E  X 2   2 mE  X   m 2
 E  X 2   2m 2  m 2
 E X 2   m 2
The second expected value rule is used to simplify the middle term and the third rule is
used to simplify the last one.
5
SMU Classification: Restricted
ALTERNATIVE EXPRESSION FOR POPULATION VARIANCE
s X2  E  X 2   m 2

s X2  E  X  m 
2

 E  X 2  2 mX  m 2 
 E  X 2   E  2mX   E m 2 
 E  X 2   2 mE  X   m 2
 E  X 2   2m 2  m 2
 E X 2   m 2
The middle term is rewritten, using the fact that E(X) and mX are just different ways of writing
the population mean of X.
6
SMU Classification: Restricted
ALTERNATIVE EXPRESSION FOR POPULATION VARIANCE
s X2  E  X 2   m 2

s X2  E  X  m 
2

 E  X 2  2 mX  m 2 
 E  X 2   E  2 mX   E m 2 
 E  X 2   2 mE  X   m 2
 E X 2   2m 2  m 2
 E X 2   m 2
Hence we get the result.
7
Introduction to Econometrics
Review of Probability
7. The Fixed and Random
Components of a Random Variable
SMU Classification: Restricted
THE FIXED AND RANDOM COMPONENTS OF A RANDOM VARIABLE
Population mean of X:
EX i   mX
In this short sequence we shall decompose a random variable X into its fixed and random
components. Let the population mean of X be mX.
1
SMU Classification: Restricted
THE FIXED AND RANDOM COMPONENTS OF A RANDOM VARIABLE
Population mean of X:
Random component In observation i:
EX i   mX
ui  X i  m X
The actual value of X in any observation will in general be different from mX. We will call the
difference ui, so ui = Xi – mX.
2
SMU Classification: Restricted
THE FIXED AND RANDOM COMPONENTS OF A RANDOM VARIABLE
Population
mean
ofof
X:X:
Population
mean
In observation
i, theInrandom
Random
component
observation i:
component is given by
Decomposition of Xi :
Hence Xi can be decomposed
into fixed and random components:
EX i   mX
ui  X i  m X
X i  m X  ui
Re-arranging this equation, we can decompose Xi as the sum of its fixed component, mX,
which is the same for all observations, and its random component, ui.
3
SMU Classification: Restricted
THE FIXED AND RANDOM COMPONENTS OF A RANDOM VARIABLE
Population mean of X:
Random component In observation i:
Decomposition of Xi :
Expected value of ui is zero:
EX i   mX
ui  X i  m X
X i  m X  ui
E ui   E  X i  m X 
 E  X i   E  m X 
 mX  mX  0
The expected value of the random component is zero. It does not systematically tend to
increase or decrease X. It just makes it deviate from its population mean.
4
SMU Classification: Restricted
THE FIXED AND RANDOM COMPONENTS OF A RANDOM VARIABLE
Population mean of X:
Random component In observation i:
Decomposition of Xi :
Variance of X is same as variance of u:
EX i   mX
ui  X i  m X
X i  m X  ui

s X2  E  X i  m X 
2
 E u 2 

s u2  E ui  0 
2


 E u 2 
The variance of X is equal to the variance of u. This is obvious, since all the variation in X is
caused by the variation in u.
5
Introduction to Econometrics
Review of Probability
8. Continuous random variable
SMU Classification: Restricted
CONTINUOUS RANDOM VARIABLES
probability
1
36
2
2
__
36
3
__
36
4
__
36
5
__
36
6
__
36
5
__
36
4
__
36
3
__
36
2
__
36
3
4
5
6
7
8
9 10 11 12
1
36
X
A discrete random variable is one that can take only a finite set of values. The sum of the
numbers when two dice are thrown is an example.
1
SMU Classification: Restricted
CONTINUOUS RANDOM VARIABLES
probability
1
36
2
2
__
36
3
__
36
4
__
36
5
__
36
6
__
36
5
__
36
4
__
36
3
__
36
2
__
36
3
4
5
6
7
8
9 10 11 12
1
36
X
Each value has associated with it a finite probability, which you can think of as a ‘packet’ of
probability. The packets sum to unity because the variable must take one of the values.
2
SMU Classification: Restricted
CONTINUOUS RANDOM VARIABLES
height
55
60
65
70
75
X
However, most random variables encountered in econometrics are continuous. They can
take any one of an infinite set of values defined over a range (or possibly, ranges).
3
SMU Classification: Restricted
CONTINUOUS RANDOM VARIABLES
height
55
60
65
70
75
X
As a simple example, take the temperature in a room. We will assume that it can be
anywhere from 55 to 75 degrees Fahrenheit with equal probability within the range.
4
SMU Classification: Restricted
CONTINUOUS RANDOM VARIABLES
height
55
60
65
70
75
X
In the case of a continuous random variable, the probability of it being equal to a given finite
value (for example, temperature equal to 55.473927) is always infinitesimal.
5
SMU Classification: Restricted
CONTINUOUS RANDOM VARIABLES
height
55
60
65
70
75
X
For this reason, you can only talk about the probability of a continuous random variable
lying between two given values. The probability is represented graphically as an area.
6
SMU Classification: Restricted
CONTINUOUS RANDOM VARIABLES
height
55 56
60
65
70
75
X
For example, you could measure the probability of the temperature being between 55 and
56, both measured exactly.
7
SMU Classification: Restricted
CONTINUOUS RANDOM VARIABLES
height
0.05
55 56
60
65
70
75
X
Given that the temperature lies anywhere between 55 and 75 with equal probability, the
probability of it lying between 55 and 56 must be 0.05.
8
SMU Classification: Restricted
CONTINUOUS RANDOM VARIABLES
height
0.05
55 56 57
60
65
70
75
X
Similarly, the probability of the temperature lying between 56 and 57 is 0.05.
9
SMU Classification: Restricted
CONTINUOUS RANDOM VARIABLES
height
0.05
55 56 5758
60
65
70
75
X
And similarly for all the other one-degree intervals within the range.
10
SMU Classification: Restricted
CONTINUOUS RANDOM VARIABLES
height
0.05
55 565758
60
65
70
75
X
The probability per unit interval is 0.05 and accordingly the area of the rectangle
representing the probability of the temperature lying in any given unit interval is 0.05.
11
SMU Classification: Restricted
CONTINUOUS RANDOM VARIABLES
height
0.05
55 565758
60
65
70
75
X
The probability per unit interval is called the probability density and it is equal to the height
of the unit-interval rectangle.
12
SMU Classification: Restricted
CONTINUOUS RANDOM VARIABLES
f(X) = 0.05 for 55  X  75
f(X) = 0 for X < 55 and X > 75
height
0.05
55 565758
60
65
70
75
X
Mathematically, the probability density is written as a function of the variable, for example
f(X). In this example, f(X) is 0.05 for 55 < X < 75 and it is zero elsewhere.
13
SMU Classification: Restricted
CONTINUOUS RANDOM VARIABLES
f(X) = 0.05 for 55  X  75
f(X) = 0 for X < 55 and X > 75
probability
density
f(X)
0.05
55 565758
60
65
70
75
X
The vertical axis is given the label probability density, rather than height. f(X) is known as
the probability density function and is shown graphically in the diagram as the thick black
line.
14
SMU Classification: Restricted
CONTINUOUS RANDOM VARIABLES
f(X) = 0.05 for 55  X  75
f(X) = 0 for X < 55 and X > 75
probability
density
f(X)
0.05
55
60
65
70
75
X
Suppose that you wish to calculate the probability of the temperature lying between 65 and
70 degrees.
15
SMU Classification: Restricted
CONTINUOUS RANDOM VARIABLES
f(X) = 0.05 for 55  X  75
f(X) = 0 for X < 55 and X > 75
probability
density
f(X)
0.05
55
60
65
70
75
X
To do this, you should calculate the area under the probability density function between 65
and 70.
16
SMU Classification: Restricted
CONTINUOUS RANDOM VARIABLES
f(X) = 0.05 for 55  X  75
f(X) = 0 for X < 55 and X > 75
probability
density
f(X)
0.05
55
60
65
70
75
X
Typically you have to use the integral calculus to work out the area under a curve, but in
this very simple example all you have to do is calculate the area of a rectangle.
17
SMU Classification: Restricted
CONTINUOUS RANDOM VARIABLES
f(X) = 0.05 for 55  X  75
f(X) = 0 for X < 55 and X > 75
probability
density
f(X)
0.05
0.25
55 56
60
65
70
75
X
The height of the rectangle is 0.05 and its width is 5, so its area is 0.25.
18
SMU Classification: Restricted
CONTINUOUS RANDOM VARIABLES
probability
density
f(X)
65
70
75
X
Now suppose that the temperature can lie in the range 65 to 75 degrees, with uniformly
decreasing probability as the temperature gets higher.
19
SMU Classification: Restricted
CONTINUOUS RANDOM VARIABLES
probability
density
f(X)
0.20
0.15
0.10
0.05
65
70
75
X
The total area of the triangle is unity because the probability of the temperature lying in the
65 to 75 range is unity. Since the base of the triangle is 10, its height must be 0.20.
20
SMU Classification: Restricted
CONTINUOUS RANDOM VARIABLES
f(X) = 1.50 – 0.02X for 65  X  75
f(X) = 0 for X < 65 and X > 75
probability
density
f(X)
0.20
0.15
0.10
0.05
65
70
75
X
In this example, the probability density function is a line of the form f(X) = b1 + b2X. To pass
through the points (65, 0.20) and (75, 0), b1 must equal 1.50 and b2 must equal -0.02.
21
SMU Classification: Restricted
CONTINUOUS RANDOM VARIABLES
f(X) = 1.50 – 0.02X for 65  X  75
f(X) = 0 for X < 65 and X > 75
probability
density
f(X)
0.20
0.15
0.10
0.05
65
70
75
X
Suppose that we are interested in finding the probability of the temperature lying between
65 and 70 degrees.
22
SMU Classification: Restricted
CONTINUOUS RANDOM VARIABLES
f(X) = 1.50 – 0.02X for 65  X  75
f(X) = 0 for X < 65 and X > 75
probability
density
f(X)
0.20
0.15
0.10
0.05
65
70
75
X
We could do this by evaluating the integral of the function over this range, but there is no
need.
23
SMU Classification: Restricted
CONTINUOUS RANDOM VARIABLES
f(X) = 1.50 – 0.02X for 65  X  75
f(X) = 0 for X < 65 and X > 75
probability
density
f(X)
0.20
0.15
0.10
0.05
65
70
75
X
It is easy to show geometrically that the answer is 0.75. This completes the introduction to
continuous random variables.
24