Lecture 1 – Week 1
Functions of Random Variables (Transformations)
Study:
Rice - Chapter 2.3 and Chapter 3.6
Wackerly - Chapter 6.4 and 6.6
Functions of random variables:
Sometimes it is necessary to find the distribution of a function of a random variable or a function
with more than one variable. Usually we find the density and then use theory to find the distribution
of the new random variable(s) and sometimes the new density is in the form of a familiar, a known
density such as the gamma or normal or beta.
Example of such questions.
If a random variable Y with a probability density function (pdf) is given by
Hence, we might be interested in a new variable and its density such as
Study the examples of Section 6.4 in Wackerly.
Applying Proposition B
UnivariateExample:
Let the random variable 𝑌 possess a uniform distribution on the interval (0,1).
a. Use the transformation method to find the pdf (probability density function) of the random
variable 𝑊 = √𝑌. What is the name of this distribution? Find the expected value of W.
b. Find the cdf (probability distribution) of 𝑊 = √𝑌.
1
Solution:
Uniform density,
𝑓(𝑦) = 1, 0 < 𝑦 < 1.
a. If 𝑊 = √𝑌 then 𝑌 = 𝑊 2 . Hence,
𝑑
= 2𝑤 .
𝑑𝑤
Applying Proposition B
𝑓𝑊 (𝑤) = 𝑓𝑌 (𝑤 2 ) × |2𝑤|
= 1 × 2𝑤
= 2𝑤
The interval of 𝑤 is
0<𝑦<1
0 < 𝑤2 < 1
0 < 𝑤 < 1.
The probability density function of random variable 𝑊 is
2𝑤,
𝑓(𝑤) = {
0,
𝟎 < 𝒘 < 𝟏,
𝑒𝑙𝑠𝑒𝑤ℎ𝑒𝑟𝑒.
Can we identify this distribution? This must always be attempted. Some guidelines:
•
•
If the boundaries of the random variable 𝑌 is between 0 ≤ 𝑦 ≤ 1 then evaluate for either an
uniform of beta probability distribution
If the boundaries of the random variable 𝑌 is between 0 ≤ 𝑦 ≤ ∞ then evaluate for a
gamma, exponential of chi-square probability distribution.
Yes, by the boundaries it might be a beta.
We rewrite the variable factor of 𝑓(𝑤)
𝑓𝑊 (𝑤) = 2𝑤(1 − 𝑤)0 .
Then we have
𝛼 − 1 = 1 𝑎𝑛𝑑 𝛽 − 1 = 0.
Hence
𝛼 = 2 𝑎𝑛𝑑 𝛽 = 1
The constant factor
Γ(𝛼 + 𝛽) Γ(2 + 1) Γ(3) 2Γ(2)
=
=
=
= 2.
Γ(𝛽)Γ(𝛼) Γ(1)Γ(2) Γ(2)
Γ(2)
Thus 𝑊~𝑏𝑒𝑡𝑎(2,1).
b. Find the distribution of 𝑊 = 𝑌 2 ,
2
𝐹𝑊 (𝑤) = 𝑃(𝑊 ≤ 𝑤) = 𝑃(√𝑌 ≤ 𝑤) = 𝑃(𝑌 ≤ 𝑤 2 )
Then
𝑤2
1. 𝑑𝑡 = 𝑤 2
𝐹(𝑤) = ∫
0
Thus the cdf of random variable 𝑊
0,
𝑤 ≤ 0,
𝐹(𝑤) = {𝑤 2 , 0 < 𝑤 < 1,
1,
𝑤 ≥ 1.
Check:
𝑓(𝑤) =
𝑑
𝑑 2
𝐹(𝑤) =
𝑤 = 2𝑤.
𝑑𝑤
𝑑𝑤
3
Convolution
4
The Multivariate Transformation Method (General Case)
Note: 𝐽 𝑖𝑠 𝑡ℎ𝑒 𝐽𝑎𝑐𝑜𝑏𝑖𝑎𝑛.
5
Example:
This Example of Rice must also be studied. Page 103. But I am doing it my way - study this.
Solution:
Normal: 𝑛~(0,1) 𝑖𝑓 𝑛~(2,3)
Standard Normal
It is given that
𝑿𝟏 𝒂𝒏𝒅 𝑿𝟐 𝒂𝒓𝒆 𝒊𝒏𝒅𝒆𝒑𝒆𝒏𝒅𝒆𝒏𝒕 and both are 𝒏(𝟎, 𝟏) distributed.
First let us have the pdf of the bivariate normal distribution. That is the joint distribution of
𝑿𝟏 𝒂𝒏𝒅 𝑿𝟐 that are 𝒏(𝝁, 𝝈𝟐 ) distributed.
𝑓(𝑥1 , 𝑥2 ) =
1
exp [−
2𝜋𝜎1 𝜎2 √1 − 𝜌2
𝑥2 − 𝜇2 2
+ (
) }]
𝜎2
1
𝑥1 − 𝜇1 2
𝑥1 − 𝜇1 𝑥2 − 𝜇2
{(
−
2𝜌
)
(
)(
)
2(1 − 𝜌2 )
𝜎1
𝜎1
𝜎2
The bivariate normal distribution:
So
𝑋1 𝑋2 ~𝑛(𝜇1 , 𝜇2 , 𝜎12 , 𝜎22 , 𝜌)
If each variable has the standard normal distribution 𝜇1 = 𝜇2 = 𝟎 and 𝝈𝟐𝟏 = 𝝈𝟐𝟐 = 𝟏 AND If
𝑿𝟏 𝒂𝒏𝒅 𝑿𝟐 𝒂𝒓𝒆 𝒊𝒏𝒅𝒆𝒑𝒆𝒏𝒅𝒆𝒏𝒕 , 𝒕𝒉𝒆𝒏 𝝆 = 𝟎
Then the above density becomes, for the standard normal variates,
6
𝑓(𝑥1 , 𝑥2 ) =
1
1
exp [− {𝑥1 2 + 𝑥2 2 }]
2𝜋
2
If 𝑿𝟏 = 𝒀𝟏 and 𝑿𝟐 = 𝒀𝟐 − 𝒀𝟏 THEN the inverses are
𝑌1 = 𝑋1 and 𝑌2 = 𝑋1 + 𝑋2 as already given by Rice and therefore
𝐽=|
𝑑 𝑦1 /𝑑𝑥1
𝑑 𝑦2 /𝑑𝑥1
𝑑 𝑦2 /𝑑 𝑥2
1
|=|
𝑑 𝑦2 /𝑑 𝑥2
1
0
|=1
1
Use
to find 𝒇(𝒚𝟏 , 𝒚𝟐 ) the joint density when we
Substitute 𝑦1 = 𝑥1 and 𝑦2 = 𝑥1 + 𝑥2 into
𝑓(𝑥1 , 𝑥2 ) =
1
1
exp [− {𝑥1 2 + 𝑥2 2 }]
2𝜋
2
so that
𝑓(𝑦1 , 𝑦2 ) =
1
1
exp [− {𝑦1 2 + (𝑦2 − 𝑦1 )2 }] × |𝐽|
⏟
2𝜋
2
=1
Simplify
1
[− {𝑦1 2 + (𝑦2 − 𝑦1 )2 }]
2
1
= [− {𝑦1 2 + 𝑦22 − 2𝑦1 𝑦2 + 𝑦1 2 }]
2
Hence the joint density of 𝑌1 𝑎𝑛𝑑 𝑌2 is
𝑓(𝑦1 , 𝑦2 ) =
1
1
exp [− {2𝑦1 2 + 𝑦22 − 2 𝑦1 𝑦2 }]
2𝜋
2
How do we recognise this as bivariate standard normal distribution?
We have to recognize all the parameters of a bivariate normal distribution.
We need to find all five parameters of the joint density of the two variables.
𝜇1 ,
Where 𝜌 =
𝜇2 ,
𝜎12 ,
𝐶𝑜𝑣(𝑌1 ,𝑌2 )
𝜎1 ,𝜎2
Given:
7
𝜎22 𝑎𝑛𝑑 𝜌
If 𝑋1 = 𝑌1 and 𝑋2 = 𝑌2 − 𝑌1 then 𝑌1 = 𝑋1 and 𝑌2 = 𝑋1 + 𝑋2
𝐸(𝑌1 ) = 𝐸( 𝑋1 ) = 0 = 𝜇1
𝐸(𝑌2 ) = 𝐸(𝑋1 + 𝑋2 ) = 𝐸(𝑋1 ) + 𝐸(𝑋2 ) = 0 = 𝜇2
𝑉𝑎𝑟(𝑌1 ) = 𝑉𝑎𝑟( 𝑋1 ) = 1 = 𝜎12 ,
𝑉𝑎𝑟(𝑌2 ) = 𝑉𝑎𝑟(𝑋1 + 𝑋2 ) = 𝑉𝑎𝑟(𝑋1 ) + 𝑉𝑎𝑟(𝑋2 ) = 1 + 1 = 2 = 𝜎22
Then 𝜎1 = 1 𝑎𝑛𝑑 𝜎2 = √2
We also want 𝑪𝒐𝒗(𝒀𝟏 , 𝒀𝟐 ) that is in order to find 𝝆. In (Chapter 5 Wackerly) we have
𝑉𝑎𝑟(𝑌2 − 𝑌1 ) = 𝑉𝑎𝑟(𝑌2 ) + 𝑉𝑎𝑟(𝑌1 ) − 2𝐶𝑜𝑣(𝑌1 , 𝑌2 )
2𝐶𝑜𝑣(𝑌1 , 𝑌2 ) = 𝑉𝑎𝑟(𝑌2 ) + 𝑉𝑎𝑟(𝑌1 ) − 𝑉𝑎𝑟(𝑌2 − 𝑌1 )
= 𝑉𝑎𝑟(𝑋1 + 𝑋2 ) + 𝑉𝑎𝑟(𝑋1 ) − 𝑉𝑎𝑟(𝑋1 + 𝑋2 − 𝑋1 )
= 𝑉𝑎𝑟(𝑋1 + 𝑋2 ) + 𝑉𝑎𝑟(𝑋1 ) − 𝑉𝑎𝑟(𝑋2 )
2𝐶𝑜𝑣(𝑌1 , 𝑌2 ) = 2 + 1 − 1
𝐶𝑜𝑣(𝑌1 , 𝑌2 ) = 1
The correlation coefficient 𝝆
𝜌=
1
1×√2
=
1
√2
so that we have
1 − 𝜌2 = 1 −
√1 −
𝜌2
= √1 − (
1 1
=
2 2
2
1
√2
1
2
) =√
Hence, the density function of a 5 parameter bivariate standard normal distribution
𝑓(𝑦1 , 𝑦2 ) =
1
𝑦1 − 𝜇1 2
𝑦1 − 𝜇1 𝑦2 − 𝜇2
exp [−
{(
−
2𝜌
)
(
)(
)
2(1 − 𝜌2 )
𝜎1
𝜎1
𝜎2
2𝜋𝜎1 𝜎2 √1 − 𝜌2
𝑦2 − 𝜇2 2
+ (
) }]
𝜎2
1
can be changed into the determined 𝑓(𝑦1 , 𝑦2 ) with the determined parameters
𝜇1 = 0,
𝑓(𝑦1 , 𝑦2 ) =
𝜎12 = 1,
𝜇2 = 0,
1
1
2𝜋(1)√2(√2)
exp [−
1
1
2(2)
𝜎22 = 2 𝑎𝑛𝑑 𝜌 =
1
𝑦2
√2
√2
{(𝑦1 )2 − 2 ( ) ( 𝑦1 ) (
8
1
√2
𝑦
2
) + ( 2 ) }]
√2
1
=
2𝜋
=
1
𝑦2
√2
√2
exp [− {(𝑦1 )2 − 2 ( ) ( 𝑦1 ) (
1
2𝜋
exp [− {𝑦1 2 − ( 𝑦1 )( 𝑦2 ) +
𝑦2 2
2
𝑦
2
) + ( 2 ) }]
√2
}]
Thus, we have our joint pdf of 𝑌1 𝑎𝑛𝑑 𝑌2 as before
𝑓(𝑦1 , 𝑦2 ) =
Hence, 𝑌1 𝑌2 ~𝑛(0,0,1,2,
1
√2
1
1
exp [− {2𝑦1 2 + 𝑦22 − 2 𝑦1 𝑦2 }]
2𝜋
2
) or we say that the pdf of 𝑓(𝑦1 , 𝑦2 ) is that of a standard normal
probability distribution with parameter values 𝜇1 = 0, 𝜇2 = 0, 𝜎12 = 1, 𝜎22 = 2 𝑎𝑛𝑑 𝜌 =
1
√2
.
Another example:
Let 𝒀𝟏 and 𝒀𝟐 be independent exponentially distributed random variables with parameter 𝝀 =
𝟏. Consider the transformations
𝒀𝟏 = 𝑿𝟏 − 𝑿𝟐 and 𝒀𝟐 = 𝑿𝟏 + 𝑿𝟐
Find the marginal density of 𝑋2 .
I have typed the solution using 𝑋1 𝑎𝑛𝑑 𝑋2 but for writing and typing I would from now on rather use
𝑈 𝑎𝑛𝑑 𝑉.
Solution:
𝑓(𝑦1 ) = 𝑒 −𝑦1
𝑦1 > 0 and 𝑓(𝑦2 ) = 𝑒 −𝑦2
𝑦2 > 0
By independence there joint pdf
𝑓(𝑦1, , 𝑦2 ) = 𝑒 −𝑦1 × 𝑒 −𝑦2 = 𝑒 −(𝑦1 +𝑦2 )
𝑦1 , 𝑦2 > 0
(due to independence)
Let 𝑦1 = 𝑥1 − 𝑥2 and 𝑦2 = 𝑥1 + 𝑥2 . Then
𝐽=
𝑑𝑦1
𝑑𝑦1
𝑑𝑥
|𝑑𝑦21
𝑑𝑥2
𝑑𝑦2 |
𝑑𝑥1
𝑑𝑥2
=|
1 −1
|=2
1 1
𝑓(𝑥1, , 𝑥2 ) = 𝑓𝑦1 ,𝑦2 (𝑥1 − 𝑥2 , 𝑥1 + 𝑥2 ) × 2
= 𝑒 −(𝑥1 −𝑥2 +𝑥1 +𝑥2 ) × 2
= 𝑒 −(2𝑥1 ) × 2
= 2 𝑒 −2𝑥1
𝑥1 − 𝑥2 > 0 and
𝑥1 + 𝑥2 > 0
Intervals:
𝑥2 < 𝑥1 and
𝑥1 + 𝑥2 > 0
Looking at the bounds we see that both 𝑥2 𝑎𝑛𝑑 𝑥1 are always larger than zero but 𝑥1 is
always larger than 𝑥2 , then this bound can be stated as 0 < 𝑥2 < 𝑥1 < ∞
𝑓(𝑥1, , 𝑥2 ) = 2 𝑒 −2𝑥1 is an exponential distribution with parameter 𝜆 = 2 or 𝛽 = 1/2,
𝑓(𝑥1, , 𝑥2 ) =
1
1/2
𝑒
−
1
𝑥
1/2 1
.
Find the marginal distribution if 𝑥2 .
9
∞
𝑓(𝑥2 ) = ∫𝑥 2 𝑒 −2𝑥1 𝑑𝑥1
2
𝑓(𝑥2 ) = 𝑒−2𝑥2
0 < 𝑥2 < 𝑥1 < ∞.
________________________________________________________________________
End of Lecture
10