Fundamentals of Data
Analysis
Lecture 12
Methods of parametric
estimation
Programm for today
Definitions
Maximum
likelihood method
Least square method
Definitions
• Estimator - any function used to estimate the unknown
parameter of the general population;
• Unloaded estimator - estimator for which the average
value is equal to zero, ie, the estimator estimating the
distribution parameter without bias;
• Efficient estimator - estimator with variance as small as
possible;
• Compliant estimator- the estimator that is stochastically
converges to the parameter the estimator that is subject
to the action of the law of large numbers (using larger
samples improves the accuracy of the estimate);
• Sufficient estimator - the estimator gathering together all
the information about tested parameter included in the
sample;
Definitions
• Point estimation - an unknown parameter estimation
method consists in the fact that as the value of the
parameter estimator is the value of this parameter
obtained from the n-element random sample;
• Interval estimation - estimation consists in the
construction of confidence intervals for this parameter.
The confidence interval is a range defined by a random
distribution of the estimator, and having the property that
covers the value of the probability defined in advance, it
usually is written in the form P(a<X<b) = 1- α.
Maximum likelihood
method
This method allows you to find estimators of unknown
parameters of such distributions in the population, which its
functional form is known.
Estimates obtained by maximum likelihood have many
desirable properties. The three most important for practical
reasons are:
1. For a large number of measurements estimator is
normally distributed;
2. Variance of the estimator, the evaluation of the accuracy
of determination of the true, is the best that can be
achieved in a given situation (optimal);
3. Estimator obtained by this method does not depend on
whether the maximum reliability calculated for the
estimated parameter, or for any of its functions.
Maximum likelihood
method
The maximum likelihood estimate of parameter
vector is obtained by maximizing the likelihood
function. The likelihood contribution of an
observation is the probability of observing the data.
The likelihood function will vary depending on
whether it is completely observed, or if not, then
possibly truncated and/or censored.
In many cases, it is more straightforward to
maximize the logarithm of the likelihood function.
Maximum likelihood
method
In the case where we have no truncation and no
censoring and each observation Xi, for i = 1 … n, is
recorded, the likelihood function is
 f(x i , ) for a population with a continuous distributi on
L(x,  ) = 
p(x i , ) for a population with the stepper distributi on
where f(xi , ) means probability density function
and p(xi ,) probability function, while  may be a
single parameter or a vector.
Maximum likelihood
method
The corresponding log-likelihood function is
n
ln L =  ln f ( xi , Θ )
i
Maximum likelihood
method
Algorithm for finding the most reliable estimator the
parameter Θ is as follows:
1. find the likelihood function L for a given
distribution of the population;
2. calculate the logarithm of the likelihood function;
3. using extreme prerequisite solve the equation:
ln L = 0
obtaining estimator:
Θ = g X 
4. check the sufficient condition for a maximum:
ln L  0
Maximum likelihood
method
The maximum likelihood method introduced
credibility intervals for appropriate levels of
reliability. The solution of the equation:
 

ln L = ln L   a
because of Θ for a = 0.5, 2, 4.5, determined
intervals corresponding to the reliability of the
reliability levels of 68%, 95% and 99.7%
Maximum likelihood
method
Example
The general population has a two-point distribution
of zero-one with an unknown parameter p. Find the
most reliable estimator of the parameter p for nelement simple sample .
Since the probability distribution of the data is a
function of:
P( xi , p) = p 1  p 
xi
1 xi
Maximum likelihood
method
Example
Therefore, the likelihood function is as follows
n
n
 xi
L   Pxi , p   p i 1
i 1
n
1  p ni1xi
 p m 1  p nm
where m is the number of successes in the sample.
ln L = m ln(p) + (n-m) ln(1-p)
Maximum likelihood
method
Example
and the differential of this expression amounting to:
 ln L  m n  m m  pn
 

p
p 1 p
p1  p 
is zero if:
pˆ 
m
n
Maximum likelihood
method
Example
The second derivative of the logarithm:
 2 ln L  m n  m 
 2 
2
p
p
1  p 2
is less than zero for p*, which means that the
reliability of the function has a maximum at that
point, and p* is the most reliable estimator of the
parameter p
Maximum likelihood
method
Exercise
The speed of sound in air measured with two
different methods is:
v1 = 340±9 m/s, v2 = 350 ±18 m/s
Find the best estimate of the speed of sound.
Note: The speed of sound is a weighted average of
these results.
Least square method
At the base of the method of least squares
is the principle according to which the
degree of non-compliance is measured by
the sum of the squared deviations of the
actual value y and the calculated Y:
(y - Y)2 = minimum.
Least square method
Found parabolic equation to the
experimental data presented in Table:
Exercise
i
1
xi
2.5
yi
6.5
x i2
6.25
xiyi
16.25
x13
15.625
xi4
39.0625
xi2yi
40.625
2
3.0
9.4
9.00
28.20
27.000
81.000
84.600
3
3.5
12.7
12.25
44.45
42.875
150.0625
155.575
4
4.0
17.0
16.00
68.00
64.000
256.0000
272.000
5
4.5
20.8
20.25
93.60
91.125
410.0625
421.200
6
5.0
26.2
25.00
131.00
125.000
625.0000
655.000
7
5.5
30.9
30.25
169.96
166.375
915.0625
934.725

28.0
123.5
119.0
551.45
532.000
2476.25
2563.725
Thank you for attention !