Order statistics
Outline
What are Order statistics (OS)?
Importance of OS
What is the difference between a random sample and
OS?
Distributions of a single OS
The joint density function of rth and sth OS?
OS in parallel and series systems
What are Order statistics?
Many functions of random variables of interest in practice depend
on the relative magnitudes of the observed variables. For instance,
we may be interested in the fastest time in a car race or the
heaviest mouse among those fed on a certain diet.
Thus, we often order observed random variables according to their
magnitudes. The resulting ordered variables are called order
statistics.
What are Order statistics?
Suppose we have a set of random variables X1, X2, …, X5 representing the
duration (in minutes) that each car takes in a car race to reach the finish line
=
X1
X2
X3
X4
X5
16
15
17
20
18
After
order
X(1)
X(2)
….
X(5)
The minimum observation = min(X1, X2, …, X5 ). = X2=15
The second minimum
= X1 =16
….
The maximum observation = max (X1, X2, …, X5 ). = X4=20
What are Order statistics?
Consider a random sample of size n; X1, X2, …, Xn
chosen from
a population with probability distribution function F(x). The
ordered variables X(1), X(2), …, X(n) are called order statistics such
that X(1) ≤ X(2) ≤ … ≤ X(n).
X(i) is called the i-th ordered statistic (i = 1, 2, … , n).
Outline
What are Order statistics (OS)?
Importance of OS
What is the difference between a random sample and
OS?
Distributions of a single OS
The joint density function of rth and sth OS?
OS in parallel and series systems
Importance of OS
Many functions of random variables of interest in practice depend on the
relative magnitudes of the observed variables. Thus, OS have a wide range of
applications in different fields such as actuarial science, reliability, quality
control, finance and many other fields. Some examples are listed below
❑ Joint life Insurance: A policy for a couple that pays out when the
first of the spouses dies. we could be interested in the minimum of the wo
lifespans of the couple.
❑ Last Survivor Policy: An insurance policy that covers a married couple
and pays the death benefit on the death of the second spouse. Second to die
insurance helps the heirs of the married couple rather than either the
husband or the wife. You want to know the distribution of the maximum of
the two lifespans.
Importance of OS (cont.)
❑ Reliability: A machine may run on 5 batteries and shuts off when the
5th battery dies. We could be interested in the distribution of the
lifetime of the third to die out of five or the distribution of the last one
to die depending on the action we are going to take.
❑ Order statistics give rise to various useful measures such as
1) The range: defined as the difference as the difference between the
maximum and the minimum value
2) The median:
Outline
What are Order statistics (OS)?
Importance of OS
What is the difference between a random sample and
OS?
Distributions of a single OS
The joint density function of rth and sth OS?
OS in parallel and series systems
What is the difference between a random
sample and OS?
A Random Sample
Random sampling assumes that the sample is taken in such a way that the
random variables for each trial are iid (independent and identically
distributed). The joint density function in this case is the product of the
common marginal densities.
Definition 1
The set of random variables X1, X2, …, Xn is said to be a random sample
of size n from a population with density function f(x) if the joint pdf has
the form. f(X1, X2, …, Xn ) = f(x1)f(x2) … f(xn).
What is the difference between a random
sample and OS?
Random sample
Order statistics
• Set of variables X1, X2, …, Xn , Xi is the
ith random variable
• Set of order variables X(1), X(2), …, X(n),
X(i) is the ith order statistics.
• Xi’s are independent and identically
distributed (iid).
• X(i)’s are not iid
• FX(x)= P(X≤ x) → The cumulative
distribution function (cdf) for X
• fX(x) → The probability density function
(pdf) for X
• FX,Y(x,y)= P(X ≤ x, Y ≤ y) → the joint
cdf of X and Y
• E(𝑋 𝑘 )→ The kth raw moment of X
• F( r )(x)= P(X( r ) ≤ x) → The cumulative
distribution function (cdf) for X(r )
• f ( r )(x )→ The probability density
function (pdf) for X (r )
• F(r ) (s) (x,y)= P(X( r ) ≤ x, X( s ) ≤ y) → the
joint cdf of X( r ) and X( s )
• E(X( r ) 𝑘 )→ The kth raw moment of X( r )
Outline
What are Order statistics (OS)?
Importance of OS
What is the difference between a random sample and
OS?
Distributions of a single OS
The joint density function of rth and sth OS?
OS in parallel and series systems
Distributions of a single OS
I. The probability density function (pdf) of the maximum (X(n))
Consider a random sample of size n; X1, X2, …, Xn each with cdf F(x). The
cdf of the nth order statistic, X(n) is given by
F(n) (x) = P( X(n) ≤ x)
= P (All X(i)’s ≤ x)
(independent)
= P( X1 ≤ x). P( X2 ≤ x) ….. . P(Xn ≤ x)
= 𝐹𝑋1 𝑥
. 𝐹𝑋2 (𝑥) . ... . 𝐹𝑋𝑛 (𝑥)
(identically distributed)
= [FX (x)]n
The pdf for the maximum X(n) could be obtained by differentiating the cdf
w.r.t x
f(n)(x)= n [FX (x)]n-1 fX (x)
Distributions of a single OS
II. The probability density function (pdf) of the minimum (X(1))
Consider a random sample of size n; X1, X2, …, Xn each with cdf F(x). The cdf of
the 1st order statistic, X(1) is given by
F(1) (x) = P( X(1) ≤ x) = 1- P(X(1) > x)
= 1- P( min (X1, X2, …, Xn ) > x)
= 1- P (All X(i)’s > x)
(independent)
= 1- [P( X1 > x|). P (X2 > x) … P( Xn > x) ]
=1- [(1 − 𝐹𝑋1 𝑥 ).(1 − 𝐹𝑋2 𝑥 )....(1 − 𝐹𝑋𝑛 𝑥 )] (identically distributed)
= 1- [1- FX (x)]n
The pdf for the minimum X(1) could be obtained by differentiating the previous cdf
w.r.t x
f(1)(x)= n [1-FX (x)]n-1 fX (x)
Distributions of a single OS
III. The probability density function (pdf) of the rth OS (X( r ) )
f( r )(x)=
𝑛!
(𝑟−1)!1!(𝑛−𝑟)!
[F(x)]r-1 f(x) [1-F(x)]n-r
Outline
What are Order statistics (OS)?
Importance of OS
What is the difference between a random sample and
OS?
Distributions of a single OS
The joint density function of rth and sth OS?
OS in parallel and series systems
The joint density function of rth and sth OS?
The joint density function of the rth and sth OS; X(r ) and X(s)
f(r )(s)(x,y) =
𝑛!
(𝑟−1)!1!(𝑠−𝑟−1)!1!(𝑛−𝑠)!
𝐹 𝑥
r-1f(x) [F(y)-F(X)]s-r-1
f(y) [1-F(y)]n-s.
The pdf of the joint n OS
f(1)(2)…(n)(𝑥1 , 𝑥2 , … , 𝑥𝑛 )= 𝑛! f(𝑥1 ) f(𝑥2 )… f(𝑥𝑛 )
Reading assignment: page 334 from the textbook
Example 1
Let X have the uniform distribution over the range (θ − 2, θ + 2). Find the
expected value of the median from a sample of size 3.
Solution:
f( 2)(x)= 3! F(x) f(x) (1-F(x))
θ−2 ≤ x(2) ≤ θ+2.
We write this domain θ − 2 ≤ x(1) ≤ x(2) ≤ x(3) ≤ θ + 2 when we have the
joint distribution of the three order statistics. The pdf of the Uniform dist.
X− θ+2
f(x)= ¼.
, F(X) =
4
f( 2)(x)= 6 (-
1
64
2
𝑥 +
1
32
θx -
1
64
2
θ +
1
16
)
The expected value μ2:3 = E(X2:3), is obtained by
θ+2
E(𝑥(2) ) = θ−2 𝑥 f( 2)(x) dx = θ
Example 2
Let X1,X2,X3 be a random sample from the exponential distribution with
mean 1, and X(1),X(2),X(3) be the order statistics of that random sample.
Find the probability density functions of X(1),X(2),X(3).
Solution: The exponential distribution is f(x) = λ𝑒 −λx = 𝑒 −x where 0 ≤ x.
F(x)= 1- 𝑒 −x .
The distributions for the three order statistics are
f (x)= 3 𝑒 −x (1-(1- 𝑒 −x ))2= 3 𝑒 −3x
( 1)
f( 2)(x)= 6 (1- 𝑒 −x ) 𝑒 −x (1-(1- 𝑒 −x ))= 6 (1- 𝑒 −x ) 𝑒 −2x
f (x)= 3 (1- 𝑒 −x )2 𝑒 −x
( 3)
Where 0 ≤ X(1),0 ≤ X(2),0 ≤ X(3).
Outline
What are Order statistics (OS)?
Importance of OS
What is the difference between a random sample and
OS?
Distributions of a single OS
The joint density function of rth and sth OS?
OS in parallel and series systems
OS in parallel and series systems
Example 3
Electronic components of a certain type have a length of life Y,
with probability density given by
f(y) =
1
100
𝑒
−𝑦
100
,
y. > 0
The Length of life is measured in hours. Suppose that two such
components operate independently and in series in a certain
system. Find the density function of the length of life of the
system and calculate its expected value
OS in parallel and series systems
Solution:
Because the system fails at the first component failure,
The length of life of the system =Y(1) = min(Y1, Y2),
f(1)(y)= 2 [1-F(Y)] f(y)
for an exponential distribution it is given by
f(1)(y)= 2 (1 − (1 − 𝑒
=
1
50
𝑒
−𝑦
50
−𝑦
100
))
1
100
𝑒
−𝑦
100
y>0
∞
E(Y(1)) = 0 𝑦 f(1)(y). 𝑑𝑦 = 50
Then it is expected that the length of life of the system is 50 hours.
OS in parallel and series systems
Example 4:
Suppose that the components in example 1 operate in parallel (hence,
the system does not fail until both components fail). Find the probability that
the length of life of the system is less than 30 hours
Sol:
The length of life of the system =Y(2) = max(Y1, Y2),
P (length of life of the system < 30)= P (Y(2) < 30) = F(2)(30)
F(2) (y)= [F(y)]2= [1 − 𝑒
F(2) (30)= [1 − 𝑒
−30
100 2
−𝑦
100 2
]
] = 0.0671
Outline
What are Order statistics (OS)?
Importance of OS
What is the difference between a random sample and
OS?
Distributions of a single OS
The joint density function of rth and sth OS?
OS in parallel and series systems