CHAPTER 1
SAMPLING DISTRIBUTIONS
§1.1 SAMPLING
Def 1.1 : The totality of elements which are
under discussion and about which information
is desired is called the target population.
Def 1.2 : Let X1,, Xn be random variables
having a joint density
as follows :
f
X1 ,,Xn
f
X1 ,,Xn
x ,, x
1
,,  that factors
  f  x  f  x  f  x  ,
n
1
2
n
where f   is the common density of each Xi .
Then X1,, Xn is defined to be a random sample
of size n from a population with density f   .
Note : x1,, xn are observed values (or observations)
of X1,, Xn .
Def 1.3 : Let X1,, Xn be a random sample from
a population with a density f   , then this population
is called the sample population.
Ex1.1 :
20-year-old males in the US
 target population
some large city we sampled
 sample population
Goal : Study the religious habits of 20-year-old males in the US.
Def 1.4 : Let X1,, Xn be a random sample from
a population with a density f   . The distribution of
X1,, Xn is defined to be the joint distribution of
X1,, Xn ; that is ,
f
X1 ,,Xn
x ,, x   f x  f x f x  .
n
1
1
2
n
Ex 1.2 : Suppose X is a random variable having
a Bernoulli distribution f  x   p q , x  0,1.Let
x 1x
X1 and X2 be a random sample from f   .Then
f
X1,X2
x , x   f x  f x   p
1
2
1
2
x1 x 2 2x1 x 2
q
, x1, x 2  0,1 .
Remark 1.1 : The definition of random sampling
here has ruled out sampling from a finite population
without replacement, since then, the results of
drawings are not independent.
Note : Suppose the form of a density f   ;  is known,
but the parameter  is unknown. Let X1,, Xn be a
t
random sample from f   ;  and T  (x1,, xn )
represent the value of some function which estimates
the unknown parameter . We want to determine which
function will be the best one to estimate .
Def 1.5 : Suppose X1,, Xn is a random sample
from the distribution of a random variable X. Then
any function T  (X1,, Xn ) of the sample is
t
called a statistic.
Ex1.3 : Suppose X1,, Xn is a random sample
from a population. Then
1 n
1. X   Xi is a statistic.
n i 1
2. min X1,, Xn   max X1,, Xn  is also a
statistic.
Def 1.6 : Let X be a random variable. The
rth (population) moment of X is defined as
 
  E X .In particular,if r  1, then
,
r
r
  E  X   , the mean of X .
,
1
Def 1.7 : Let X be a random variable. The
rth central moment of X about  is defined
r

by r  E  X     . In particular,if r  2, then


2

2  E  X       2, the variance of X .


Def 1.8 : Let X1,, Xn be a random sample
from a population. The rth sample moment
n
1
about 0 is defined to be Mr'  Xir . If r  1,
n i 1
1 n
we get the sample mean X  Xn  Xi .
n i 1
Also,the rth sample moment about X is defined
1 n
by Mr  (Xi  X )r .
n i 1
Thm 1.1 : Let X 1, , X n be a random sample
from a population. Then
 


1
(b )Var  M   E  X   E  X  


n
(a )E M r'  r'
'
r
if r' exists ;
2r
 
r

2




2
1 '

'
'
   2r  r  if  2r exists .
n

In particular, if r  1 ,then E  X    and
Var  X  

2
, where  and  are the mean
2
n
and variance of the population, respectively.
Def 1.9 : Let X1,, Xn be a random sample from
a population.The sample variance is defined by
n
1
2
S S 
(Xi  X ) ,n  1.

n  1 i 1
2
2
n
Thm1.2 : Let X1,, Xn be a random sample from a
population with mean  and variance  . Then
2
  
E S
2
2
 
and Var S
2
1
n 3 4 
 , n  1.
  4 
n
n 1 
where  and  2 are the mean and variance of the
population, respectively.
§1.2 SAMPLE MEAN
Thm1.3 :(Weak law of large number)
Let X1,, Xn be a random sample from a distribution
with mean E  Xi    and variance Var  Xi    2 < .


Then , for every   0, we have lim P Xn   <   1.
n
Thm1.4 :(Strong law of large number)
Let X1,, Xn be a random sample from a distribution
with mean E  Xi    and variance Var  Xi  = < .
2


Then , for every   0, we have P lim Xn   <   1.
n
Thm1.5 : (Central limit theorem; CLT)
Let X1,, Xn be a random sample from a
distribution with finite mean E  Xi    and
variance Var  Xi  = < . Then, the random
2
variable Zn =
Xn  

approaches the standard
n
normal distribution N  0,1 as n  . That is,
FZ  z   P Zn  z   
n
z
1

2
e
dz    z  .
z 2 2
§1.3 SAMPLING FROM THE NORMAL DISTRIBUTION
Thm1.6 : Let X1,, Xn be a random sample from a
2

 
2
N , .Then X  N  ,  .
 n 
Thm1.7 : Let X1,, Xn be independent random




variables and Xi  N i ,  i 2 , i  1,, n. Than
 Xi  i
U   
i
i 1 
n
2

2
   n  .

X   
2
Cor1.1 : If X  N ,  , then 
   1 .
  

2



Cor1.2 : Let X1,, Xn be a random sample from a N , 2 .
n
Then 
X  
i 1
i

2
2
  n  .
2
Thm1.8 : Let X1,, Xn be a random sample from a


N , . Then (a)X and S are independent random
2
variables. (b)
2
2

1
n
S
 

2
  n  1 .
2
Thm1.9 : Suppose U and V are independent variables
Um
2
2
and U   m and V   n  .Then X 
 F m,n  .
Vn
Cor1.3 : Let X1,, Xm be a random sample


sample from a N   ,  . Suppose the two
from a N X , , and let Y1,,Yn be a random
2
X
2
Y
Y
samples are independent. Then
SX2  X2
SY2 Y2
 F m  1, n  1 .
In particular,if    , then
2
X
2
Y
S
2
X
2
Y
S
 F m  1, n  1 .
Remark1.2 : If X  F m, n  , then
n
1.   X  =
for n  2 and
n2
2n 2 m  n  2 
for n  4.
Var  X  =
2
m n  2  n  4 
1
1
2.
 F n, m  and F m, n  
.
X
F1 n, m 
Thm1.10 : If   N  0,1 and U   2 r  .Then
X

U r
 t r  .
Remark1.3 :
(1) If r  1, then the t-distribution reduces to a Cauchy
distribution.
(2) As r increases, the t-distribution approaches the
N  0,1 .
Z
(3)X 
 F 1,r  .
Ur
2
2
Cor1.4 : Suppose that X1,, Xn is a random sample froma


N , . Then
2
X 
S n
 t n 1
X  

and
S n
2
2
 F 1,n 1 .
§1.4 ORDER STATISTICS
1.4.1
DEFINITIONS AND DISTRIBUTIONS
Def 1.10 : Let X1,, Xn be a random sample
from a population. Then Y1  Y2    Yn
X   X     X  , where Y X  ,i  1,n,
1
2
n
i
i
are the Xi arranged in order of increasing
magnitudes and are defined to be the order
statistics corresponding to X1,, Xn .
Note :
(i) Yi are statistics  functions of X1,, Xn  and
are in order.
(ii) Order statistics are NOT independent
Y  y Y
j
j 1

y .
(iii) Y1= min X1,, Xn  and Yn =max X1,, Xn  .
Thm1.11 : Let Y1,,Yn denote the order statistics
of a random sample, X1,, Xn from a c.d.f F   .
n
Then FY y   

j 
n j
  F y  1  F y 
j
n
j
.
Cor1.5 : From Theorem 1.11,we have
  F y  1  F y   F y 
y      F y  1  F y 
(a) FY y  
n
(b) FY
1
n
n
n
n
j 1
n
j
 1  1  F y  
0
j
n
nj
n
;
Thm1.12 :Let Y1,,Yn denote the order statistics
of a random sample, X1,, Xn from a continuous
population with c.d.f F   and p.d.f f   .
(a)The p.d.f of Y is
n 
 1
n!
F y   1  F y   f y  .
fY y  

 


n



!

!
1
  
(b) The joint p.d.f of Y and Y , 1      n, is

fY ,Y y ,y


 

 1
n!
F y  



!
!
n
!





1


1

 
 
 F y  F y  


  y  y  .
  1
 
n 
1  F y 
 

(c) The joint p.d.f of Y1,,Yn is
 
f y  f y , for
n ! f y1 f yn  y1  y2    yn
fY Y y1,, yn   
1
n
otherwise.
0
Def1.11 : Let Y1,,Yn be the order statistics of a
random sample, X1,, Xn , from a population.
(i) The sample median is defined by
Yn1
if n is odd
 2

M  Y Y
n
n
1
 2
2
if n is even.
 2
(ii) The sample range is defined to be R Yn Y1.
Y1 Yn
(iii) The sample midrange is defined to beT 
.
2
Remark1.4 :
(i) If n  2k  1, then Yk 1 is the sample median and
fk 1 y  is given by Theorem1.12(a).
Y

(ii) If n  2k, the sample median is
k
Yk 1 
. We
2
can obtain the distribution by a transformation
starting with the joint density of Yk and Yk 1
given by Theorem1.12(b).
Thm1.13 : If R is the sample range and T is the
sample midrange from a continuous population
with cdf F   and pdf f   . Then the joint pdf of
R and T is given by fR,T r,t   n n  1
 F t  r 2  F t  r 2  f t  r 2 f t  r 2 ,
r  0, and the marginal distributions are given by
n 2
fR r    fR,T r,t dt and fT t    fR,T r,t dr .



0
Ex1.5 : X1,, Xn  iid U  0,1
 r   r 
fR,T r, t   n n  1  t     t    1  1
 2   2 
r
r
n 2
 n n  1 r , 0 < r < 1, < t < 1 
2
2
fR r    fR,T r, t dt


r
1
2
r
2

n 2

n n  1 r dt  n n  1 r  t  r
n 2
 n n  11  r  r n 2 , 0 < r < 1.
1
2
r
2
fT t    fR,T r, t dr


 21t 
1
n 2
0 n n  1 r dr, < t < 1
2

2t
 n n  1 r n 2dr, 0 < t  1
0
2
n 1

1
n  2t  , 0 < t 
2

n 1
n 2 1  t   , 1 < t < 1

 
2
n 1
 R  Beta n  1, 2 , E  R  
n 1
E T    t  fT t  dt or
t
Y1 Yn  1
E T   E 
  E Y1   E Yn  
 2  2
n  1
1 1
 


2  n  1 n  1 2
1.4.2 ASYMPTOTIC DISTRIBUTIONS
Ex1.6 : Let Y1,,Yn denote the order
statistics of a sample of size n from U  0,  .
1
x


Then f  x   , 0 < x <  . F  x  
 y n
n
  , 0  y < 
So FY y   F y      
n

 1 ,  y< 
0, 0  y < 
Thus, lim FY y   
n
n 
1,   y < 
Ex1.7 : Yn : nth order statistics of a random sample from
U  0,  . Let Zn  n  Yn  . Then the p.d.f of is
 z  dy
hZ  z   fY    
n
n
 n  dz
n1
 z
   1 1
n  n    
    n


n1
 z
  
n


, 0< z < n
n

So the p.d.f of Zn is
0, z < 0
 z
HZ  z    hZ u du, 0  z < n
n
n
0

1, n  z
0, z < 0

n 1
u
 
n
 z   n 

z 


 
du  1  1   , 0  z < n
n
0

 n 



1, n  z < 
Hence the limiting distribution of Zn is
0, z < 0

n


HZ  lim HZ  z   


z
n
n 


lim
,



z
1
1
0


n 
n  


 

0, z < 0


z

1  e  , z  0