Chapter 6 Parameter estimation
The previous chapter has pointed out that the basic problem of mathematical statistics is
inferring the distribution of totality through sample, which is called statistical inference. The
main elements of statistical inference including parameter estimation and statistical hypothesis
testing, they constitute the core of mathematical statistics. This chapter focuses on parameter
estimation Methods and criteria for evaluation of the estimated amount, and focus on the
classical methods of point estimation as well as the normal population parameter interval
estimation.
§ 6.1 Point estimation
Population is characterized by population distribution. In practical problems, we can often
Determine the type of population distribution through relevant expertise and previous
experience, but the parameters are unknown. We often estimate the parameters by the
sample, which is known as parameter estimation.
Set the distribution function of population
X is F ( x) , which contains the unknown

parameters
 ,we call any function  ( X 1 ,..., X n )
of a sample point estimator.
1 Moment estimate
The method of moments is the oldest method of finding point estimators. If population
distribution contains k unknown parameters, method of moments estimator are found by
equating the first k sample moments to the corresponding k population moments, and solving
the resulting system of simultaneous equations.
Example 6.1 suppose ( X1 , X 2 ,..., X n ) are iid P( ) ,We have
1  X , 1  , according

to the method of moments, we yield the moments estimator of  is   X .
Example 6.2 suppose ( X1 , X 2 ,..., X n ) are iid N (  ,  ) ,
2
We have  1  X , 2 
1 n 2
 X i , 1   , 2   2   2 ,hence we must solve the equation
n i 1
  X ,

 2
1 n 2
2




 Xi ,

n i 1

Solving for
 ,  2 yields the
moments estimator of
 ,  2 is

  X ,
 2 n  1 2 1 n
S   ( X i  X )2 ,
 
n
n i 1

Example 6.3 suppose ( X1 , X 2 ,..., X n ) are iid,and the common pdf is
 1 1  (1 1 )
x
, x  c,
c
f ( x; )   

0,
其它,

0    1， c is a known positive constant，we have
1  EX 

 xf ( x; )dx




1
xc 

c

Let
  X ,we yield the
1
x
1
 (1 )

dx
c
,
1

moment estimator of
 is   1 
c
.
X
2 Maximum likelihood estimates
Consider a random sample
( X1 , X 2 ,..., X n ) from a distribution having pdf or
probability distribution  ( x; ) ,   (1 ,...,  m ) are unknown parameters, The joint
n
pdf of
( X1 , X 2 ,..., X n ) is
  ( x ; ) ,this may be regard as a function of
i
i 1
 ,when so
regarded, it is called the likelihood function L of the random sample, and we write
n
L( x1 ,..., xn ; )    ( xi ; ) ,
i 1
Suppose
that
we
can
find
a
nontrivial
function
of
( x1 , x2 ,..., xn ) ,say
u( x1 , x2 ,..., xn ) ,when  is replaced by u( x1 , x2 ,..., xn ) ,the likelihood function L is
a maximum, then the statistic u ( X1 , X 2 ,..., X n ) will be called a maximum likelihood

estimator of
 ,and will be denoted by the symbol   u ( X 1 , X 2 ,..., X n ) .
The function L can be maximized by setting the first partial derivation of
ln L ,with respect to  ,equal to zero, that is to say
  ln L
   0,
1


  ln L

 0.
  m
and solving the resulting equation for
of  .
 ,which is the
maximum likelihood estimator
If the first partial derivation of ln L is not exist, it will be solved by the
definition of maximum likelihood estimator.
Example 6.4 Let ( X1 , X 2 ,..., X n ) denote a random sample from a distribution that is

P( ) ,we shall find  , the maximum likelihood estimator of  .The probability
distribution of X is
p( x;  )  P(  x)  e  
x
x!
, x  0,1, 2,...,
the likelihood function L is
n
n
L   e 
i 1

xi
xi !
 e n

 xi
i 1
n
x !
i 1
,
i
so
n
n
i 1
i 1
ln L  ( xi ) ln  ( ln xi !)  n ,
setting
d ln L 1 n
  xi  n  0 ,
d
 i 1
Solving the equation, we can obtain that the maximum likelihood estimator of


 is
1 n
 xi  x .
n i 1
.
Example 6.5 Let ( X1 , X 2 ,..., X n ) denote a random sample from a distribution that is
N (  ,  2 ),  ,  2 are unknown parameters, we shall find ˆ , ˆ 2 , the maximum likelihood
estimators of
n
L
i 1
 ,  2 .we can easily obtain the likelihood function L is
( xi   )2
1
exp[
]  (2 )n 2 ( 2 )n 2 exp[ 2
2
2
2
2
n
1
(x
i 1
i
  )2 ] ,
So
n
n
1
LnL   ln(2 )  ln  2 
2
2
2 2
Setting
n
 (x
i 1
i
  )2 ,
1 n
 
  LnL   2 ( xi  n )  0,

i 1

n
  LnL   n  1
( xi   ) 2  0 ,
2
2 2 
  2
2
2( ) i 1
Solving the equation, we can obtain that the maximum likelihood estimator of
 ,  2 ,is
1 n



 Xi  X ,

n i 1


n
 2  1 ( X  X ) 2 .

i

n i 1
Example 6.6 If the probability distribution of X is
0
 2

1
2
3 
,
2
2 (1   )  1  2 
1
2
 ( 0    ) is unknown parameter, we observed the following eight values of X ,
3, 1, 3, 0, 3, 1, 2, 3,
Find the moment estimate and the maximum likelihood estimate of
.
From the distribution of X ,we can easily obtain that EX  3  4 ,setting 3  4  x ,we

have that the moment estimator of
1
4
 is   (3  X ) .
According to the sample values, we have x  2 ,so the moment estimate of
 is
1
.
4
the likelihood function L is
8
L( )   P( X  xi )  4 6 (1   ) 2 (1  2 ) 4 ,
i 1
so
ln L( )  ln 4  6 ln   2 ln(1   )  4 ln(1  2 ) ,
Setting
d ln L( ) 6
2
8
 

 0,
d
 1   1  2
Solving the equation, we have  
estimate of
 is
7  13
7  13 1
 , so the maximum likelihood
.because
12
12
2
7  13
.
12
Example 6.7 Let ( X1 , X 2 ,..., X n ) denote a random sample from a distribution that
is U [a, b] , Find the maximum likelihood estimators of a , b .The pdf of population is
 1
, a  x  b,

p( x)   b  a
 0,
其它,
So we can obtained the likelihood function L is
 1
, a  xi  b, i  1, 2,..., n,

L   (b  a) n
 0,
其它.

Which is an ever-decreasing function of b and an ever-increasing function of a ,The maximum
of such functions cannot be found by differentiation but by selecting b as small as possible and
a as large as possible. Now a  each xi and b  each xi ,so a  min( xi ) and
b  max( xi ) .so the maximum likelihood estimators of a , b is

 a  min( X 1 ,..., X n ),

b  max( X 1 ,..., X n ).
3, Methods of evaluating estimators
The methods discussed in the previous section have outlined reasonable techniques for
finding point estimators of parameters. A difficulty that arises,however,is that since we can
usually apply more than one of these methods in a particular situation, we are often faced with
the task of choosing between estimators. of course, it is possible that different methods of
finding estimators will yield the same answer, which makes evaluation a bit easier. but in many
cases, different methods will lead to different estimators.
3.1 Unbiased estimator


If an estimator  ( X 1 ,..., X n ) satisfies that E  ( X 1 ,..., X n )   ,the estimator is
called unbiased.
Because estimator is a random variable, different sample lead to different estimate, the
character of unbiased mean that a good estimator should equal to the parameter in mean meaning.
Example
6.8
Let
( X1 , X 2 ,..., X n )
X , EX   , DX   2 ,Prove that X and S
estimator of
denote
2

 ,  2 respect.
Proof EX  E (
1 n
1 n
1
X
)

EX i  .n    ,


i
n i 1
n i 1
n
a
random
sample
from
1 n
( X i  X ) 2 are the unbiased

n  1 i 1
DX  D(
ES
2
1 n
1
Xi )  2

n i 1
n
 E[
n
 DX
i 1
i
1
 2,
n
1 n
( X i  X )2 ]

n  1 i 1

n
1
E{ [( X i   )  ( X   )]2 }
n  1 i 1

1 n
 [ E ( X i   )2  2 E ( X i   )( X   )  E ( X   )2 ]
n  1 i 1

1 n
2
1
( 2   2   2 )

n  1 i 1
n
n
2.
In Example 6.7, we can easily know that
1
1
( X 1  X 2 ) and ( X 1  X 2  X 3 ) are
2
3
both the unbiased estimators of  ,which is better? So we should search another
standard to evaluate estimators.
3.2 Efficient estimator


Let  1 and  2 are both the unbiased estimator of

 ,if

D 1  D 2 ,


We say  1 is more efficient than  2 .If the number of sample n is fixed, and the variance of


 is less than or equal to the variance of every other unbiased estimator of  ,we say  is the
efficient estimator of
1
2
.
Because D[ ( X 1  X 2 )] 
more efficient than
1 2
1
1
1
  D[ ( X 1  X 2  X 3 )]   2 ,so ( X 1  X 2 ) is
2
3
3
2
1
( X1  X 2  X 3 ) .
3
Example 6.9 Consider a random sample ( X1 , X 2 ,..., X n ) from a distribution having pdf
 1 x 
 e , x  0,
p( x)   
 0 ,
ot her s,
Z  min( X1 ,
, X n ) , Prove that X and
nZ
 ,and when n  1, X is more efficient than nZ .
are both the unbiased estimators of
Proof
we can easily calculate that EX   ,so EX   ,that is to say X is the unbiased
estimator of
 .On the other side,
n
FZ ( z )  P( Z  z )  1  P( Z  z )  1   P( X i  z )  1  e

nz

,z  0,
i 1
n


n
We have Z ~ E ( ) ,so EZ 
of
, E (nZ )   , that is to say nZ is the unbiased estimator
 too. In order to compare the efficiency of X and nZ ,we can also easily calculate that
D  X    2 , D( X ) 
2

, D(nZ )  n 2 D ( Z )  n 2 .( ) 2   2 , when n  1,
n
n
D( X ) 
2
  2  D(nZ ) ,
n
So X is more efficient than nZ ..
3.3 Uniform estimator
If for any
  0 ,we have

lim P(| n   |  )  1 ,
n 

We call  n is the uniform estimator of  .
§6.2
Interval
estimation
Generally we have two kinds of forms to estimate a parameter, One is by a point, which is
introduced in§6.1,the other is by a interval. for example, we estimate the age of a person is
between 35 and 40;we called this kind of estimator interval estimator.
An interval estimator of a real-valued parameter  is any pair of function

1 ( X 1 ,..., X n )

 2 ( X 1 ,..., X n )
and

of
a
sample
that
satisfy
1 ( X 1 ,..., X n )

  2 ( X 1 ,..., X n ) for all sample x. if the sample x is observed, the inference




1 ( X 1 ,..., X n )     2 ( X 1 ,..., X n ) is made. The random interval (1 , 2 ) is called an


interval estimator; If P(1 ( X 1 ,..., X n )     2 ( X 1 ,..., X n ))  1   ,we call the


confidence coefficient of (1 ,  2 ) is 1   .we often set
 =0.01,0.05,0.1 etc.
1, Interval estimator for mean in normal population
1.1 Let ( X1 , X 2 ,..., X n ) denote a random sample from a distribution that is
N (  ,  2 ),  2 is known,  is unknown parameter, What is the interval estimator of 
with the confidence coefficient 1   ?
Set
U
We have U
X 
/ n
,
N (0,1), By the definition of u /2 we have
P(| U | u /2 )  1   ,
P(|
X 
/ n
| u /2 )  1   ,
P( X  u /2 . / n    X  u /2 . / n )  1   ,
the
interval
estimator
of 
with
the
confidence
coefficient 1  
is
( X  u /2 . / n , X  u /2 . / n ) .
Example 6.10 Let a random sample of size 10 from the normal distribution
X
N (  ,0.22 ), They are
18.3, 17.5, 18.1, 17.7, 17.9, 18.5, 18, 18.1, 17.8, 17.9,
Determine a 95 per cent confidence interval for  .
Solution 1    0.95 ,   0.05 , n  10,   0.2, u /2 =1.96, u /2 . /
sample, we have
n =0.124,By the
x  17.98 ,so the 95 per cent confidence interval for  is
(17.98-0.124, 17.98+0.124)= (17.856, 18.104).
1.2 Let ( X1 , X 2 ,..., X n ) denote a random sample from a distribution that is
N (  ,  2 ),  and  2 are unknown parameters, What is the interval estimator of  with
the confidence coefficient 1   ?
Set
T
X 
,
S / n
We have T ~ t (n  1) ,so
P(| T | t /2 (n  1))  1   ,
Hence the interval estimator of  with the confidence coefficient 1   is
( X  t /2 (n  1)S / n , X  t /2 (n  1)S / n ) .
Example 6.11 Let a random sample of size 9 from the normal distribution
N ( ,  2 ) ,They are
0.497,0.506，0.518，0.524，0.488，0.510，0.510，0.515，0.512，
Determine a 99 per cent confidence interval for  .
Solution 1    0.99 ,   0.01, n  9 , t /2 (8) =3.3554,From the sample data ,we have
x  0.5089 , S  0.0109 , t /2 (n  1)S / n =0.0122, so the 99 per cent confidence
interval for  is (0.5089-0.0122, 0.5089+0.0122)=(0.4967,0.5211).
2, Interval estimator for variance in normal population
Let ( X1 , X 2 ,..., X n ) denote a random sample from a distribution that is
N (  ,  2 ),  and  2 are unknown parameters, What is the interval estimator of  2
with the confidence coefficient 1   ?
Set
 
2
We have
1
2
n
( X
i 1
 X) 
2
i
(n  1) S 2
2
,
 2 ~  2 (n  1) ,so
P( 12 /2 (n  1)   2  2 /2 (n  1))  1   ,
n
 n
2
( X i  X )2

 (Xi  X )
P  i 1 2
  2  i 12
1 /2 (n  1)
  /2 (n  1)


So the interval estimator of 
2


  1 ,



with the confidence coefficient 1  
n
 n
2
(
X

X
)
( X i  X )2

 i
 i 1 2
, i 12

(
n

1)
1 /2 (n  1)
  /2


is


.



Example 6.12 Let a random sample of size 9 from the normal distribution
N ( ,  2 ) ,They are
600, 612, 598, 583, 609, 607, 592, 588, 593,
Determine a 95 per cent confidence interval for  .
2
Solution From the sample data, we can easily calculate that x  598 , S  98.5 ,
2
n
 ( x  x)
2
i
i 1
 ( n  1) S
2
=788,  0.975 (8)  2.18,  0.025 (8)  17.5 ,so the 95 per cent
2
confidence interval for 
2
788 788
,
) ,that is to say (45.03,361.47).
17.5 2.18
2
is (
3, Interval estimator for difference of means and ratio of variance in two normal
populations
Let ( X 1 , X 2 ,..., X n1 ) denote a random sample of size n1 from a distribution that
2
is N ( 1 ,  1 ), (Y1 , Y2 ,..., Yn2 ) denote a random sample of size n2 from a distribution
that is N ( 2 ,  2 ), where 1 ,  1 ,  2 ,  2 are
2
2
2
unknown parameters, the two random
samples are independent, What is the interval estimator of 1  2
and  1 /  2 with the
2
2
confidence coefficient 1   ?
3.1 Difference of means
If  1   2   ,but 
2
2
2
T
2
is unknown, set
( X  Y )  ( 1  2 )
S
,
1 1

n1 n2
So T ~ t (n1  n2  2) ,hence
P(| T | t /2 (n1  n2  2))  1   ,
So the interval estimator of 1  2 with the confidence coefficient 1   is
( X  Y  t /2 (n1  n2  2) S
1 1
1 1
 , X  Y  t /2 (n1  n2  2) S
 ).
n1 n2
n1 n2
3.2 ratio of variance
Set
F
So F
S12 S22
 12  22
,
F (n1  1, n2  1) .hence
P( F1 2 (n1  1, n2  1)  F  F 2 (n1  1, n2  1)  1   ,
So the interval estimator of  1 /  2 with the confidence coefficient 1   is
2
2
(
S1 2 S2 2
S1 2 S2 2
,
)
F 2 (n1  1, n2  1) F1 2 (n1  1, n2  1)
Example 6.13 Let a random sample of size 4 from the normal distribution
N ( 1 ,  2 ) ,They are
0.143
0.142
0.143
0.137
Another random sample of size 5 from the normal distribution N (  2 ,  ) ,They are
2
0.140
0.142
0.136
0.138
0.140
The two random samples are independent; determine the interval estimator of 1  2
and
 12 /  22 with the confidence coefficient 0.95.
Solution
From
the
sample
data,
we
can
easily
calculate
that X  0.14125, S1  0.00000825, Y  0.1392, S2  0.0000052 , t0.025 (7)  2.36 ,
2
2
F0.025 (3, 4)  9.98, F0.975 (3, 4) 
1
1

 0.0662 ,
F0.025 (4,3) 15.1
so
S 
3  0.00000825  4  0.0000052
 0.00255091 ,
452
Hence the interval estimator of 1  2
with the confidence coefficient 0.95 is
(-0.002,0.006) and the interval estimator of  1 /  2 with the confidence coefficient
2
2
0.95 is (0.159,23.957).
Exercises
1. Let ( X 1 ,
, X n ) be a random sample from the distribution having pdf
(  1) x ,0  x  1,
，
f ( x)  
0, ot her s,
Find the moment estimator and the maximum likelihood estimator of
2. Let ( X 1 ,
.
, X n ) be a random sample from the distribution having pdf
 e  x , x  0,
(   0) ，
f ( x;  )  
 0, x  0
Find the moment estimator and the maximum likelihood estimator of  .
3. Let ( X 1 ,
, X n ) be a random sample from the distribution having pdf
 3x 2
, 0  x  ,

f ( x)    3
  0，
 0,
其它

Find the moment estimator of  .
4. Let ( X 1 ,
, X n ) be a random sample from the Geometry distribution having
probability distribution
P( X  k )  p(1  p) k 1 , k  1,2,,0  p  1,
Find the maximum likelihood estimator of p .
5. Let ( X 1 ,
, X n ) be a random sample from the distribution having pdf
2e2( x  ) , x   ,
f ( x; )  
x  ,
 0,

  0 is unknown parameter. Setting   min( X 1 , , X n ) ,
where
(1) Determine the distribute function of X ;

(2) Determine the distribute function of

(3)  is the unbiased estimator of
;
 or not?
6. Let  be the mean of population, ( X 1 , X 2 , X 3 ) is a random sample from the
population distribution, show

2
1
2
X1  X 2  X 3 ,
5
5
5

1
1
1
2  X 1  X 2  X 3 ,
6
3
2

1
3
9
3  X 1  X 2  X 3 ,
7
14
14
The unbiased estimators of  ;
(1) 1 



(2)Compare the efficiency of 1 ,  2 , 3 .
7.
X
Let
a
random
sample
of
size
6
from
the
normal
distribution
N (  , 0.25), they are
14.70, 15.21, 14.90, 14.91, 15.32, 15.31,
Determine a 95 per cent confidence interval for  .
8. Let a random sample of size 9 from the normal distribution N ( ,  ) ,They
2
are
21.1, 21.3, 21.4, 21.5, 21.3, 21.7, 21.4, 21.3, 21.6,
Determine a 99 per cent confidence interval for  .
2
9. Let a random sample of size 8 from the normal distribution N ( 1 ,  ) ,They
2
are
0.143
0.142
0.143
0.137
Another random sample of size 9 from the normal distribution N (  2 ,  ) ,They are
2
0.140
0.142
0.136
0.138
0.140
The two random samples are independent; determine the interval estimator of 1  2
and  1 /  2 with the confidence coefficient 0.90.
2
2
10. Let ( X1 , X 2 ,..., X n ) denote a random sample from a distribution that is
N (  ,9) ,If the length of the interval estimator of  with the confidence coefficient
1   is no longer than 2.How many sizes of the sample at least should be in following
two situations
(1)   0.1时;
(2)   0.01时.