DEPENDENCE AND INDEPENDENCE OF RANDOM VARIABLES
NANEH APKARIAN
Dear Class,
It has come to my attention that I made an erroneous statement about determining the independence of random variables. This note is to correct that mistake, and to try to explain the true
nature of these characteristics, as used in this class.
Setup: We will perform some randomized experiment from which we will collect data on the
random variables X and Y . We wish to analyze the dependencies between these random variables.
Definition 1 (Dependence). The random variable X is dependent on Y if and only if X can be
written as a function of Y , i.e.
X = f (Y ).
The random variable Y is dependent on X if and only if Y can be written as a function of X, i.e.
Y = g(X).
Note: X = f (Y ) does not imply that Y = g(X). The function may not be invertible.
Fact 1. Not dependent DOES NOT IMPLY dependent.
Definition 2 (Independence). The variable X is independent of Y if and only if
P [X = x|Y = y] = P [X = x]
for all
x ∈ ΩX , y ∈ ΩY
Theorem 1. X independent of Y if and only if
P (X = x and Y = y) = P (X = x) · P (Y = y)
∀x ∈ ΩX , y ∈ ΩY
Theorem 2. If X is independent of Y , then Y is independent of X.
Theorem 3. For any random variables X and Y,
E[XY ] = E[X] · E[Y ] + Cov(X, Y )
Theorem 4. If X, Y are independent random variables, then Cov(X, Y ) = 0.
Corollary 5. If X, Y are independent random variables, then
E[XY ] = E[X] · E[Y ]
1
2
NANEH APKARIAN
NOTE: E[XY ] = E[X] · E[Y ] does NOT imply that X, Y are independent in general. You
must check the conditional probabilities to be sure.
To be explicit, using the notation,
P (xi , zj ) := P [X = xi and Y = yi ],
P (xi ) = P [X = xi ],
We must establish whether or not the tables
z1
z2
...
x\z
x1 P (x1 , z1 ) P (x1 , z2 ) . . .
..
.
x2 P (x2 , z1 )
..
..
.
.
xn P (xn , z1 )
...
P (zj ) = P [Z = zj ],
zm
P (x1 , zm )
..
.
P (xn , xm )
and
x\z
x1
x2
..
.
xn
are equal in every entry.
z1
z2
...
P (x1 )P (z1 ) P (x1 )P (z2 ) . . .
..
.
P (x2 )P (z1 )
..
.
P (xn )P (z1 )
...
zm
P (x1 )P (zm )
..
.
P (xn )P (xm )
DEPENDENCE AND INDEPENDENCE OF RANDOM VARIABLES
3
Example 1. A funny casino has the roulette wheel shown:
When the wheel is spun, X is given by the value of the outermost circle, Y by the intermediate
circle, and Z by the innermost. Analyze the dependencies of the pairs {X, Y }, {X, Z}, {Y, Z}.
(1) Note that
P [Y = 0] = 1/12
but
P [Y = 0|X = −3] = 0.
This tells me that X, Y are not

−3
2
X=

5
independent. Careful observation of the wheel reveals that
if Y ∈ {−4, −5, −6}
if
Y ∈ {−1, 0, 1} = f (Y )
if
Y ∈ {2, 3, 4}
Thus X is a function of Y , and so X is dependent on Y .
However, X = −3 does not uniquely determine a value for Y , we still have the options
{−4, −5, −6}. Thus Y is not dependent on X.
(2) We will use the following tables to prove that X, Z are independent. Fill out the tables:
Using the wheel, we discover that the joint probabilities P (xi , zj ) are:
x\z
-3
2
5
1
2
3
1/6 1/6 1/6
1/12 1/12 1/12
1/12 1/12 1/12
Also from the wheel, we observe/calculate that
P [X = −3] = 1/2
P [X = 2] = 1/4
P [X = 5] = 1/4
P [Z = 1] = P [Z = 2] = P [Z = 3] = 1/3,
4
NANEH APKARIAN
and so fill in the product of the probabilities:
x\z
-3
2
5
1
2
3
1/6 1/6 1/6
1/12 1/12 1/12
1/12 1/12 1/12
Note that in every entry the values are equal. THUS X, Y are independent.
(3) Work this one out for yourself.
Also, in this case, it turns out that Z = X − Y . This means that Z is dependent on {X, Y }.
However, we know that X = f (Y ), meaning that
Z = X − Y = f (Y ) − Y = g(Y )
So in reality, Z is dependent on Y alone. Nifty!