S OME E LEMENTARY S TATISTICAL I NFERENCES
Chapter 5: Some Elementary Statistical Inferences
• Sampling Statistics
• Order Statistics
• More on Confidence Intervals
• Introduction to Hypothesis Tests
S OME E LEMENTARY S TATISTICAL I NFERENCES
Sampling and Statistics
• In a typical statistical problem, we have a random variable X of
interest but its pdf f (x) or pmf p(x) is not known. But the form of
f (x) or p(x) is known down to a parameter θ (maybe a vector).
–
X has an exponential distribution, exp(θ).
–
X has b(n, p) distribution, where n is known but p is unknown.
–
X has Gamma distribution Γ(α, β), where both α and β are
unknown.
• We want to estimate θ from a random sample.
S OME E LEMENTARY S TATISTICAL I NFERENCES
An Motivation Example
• An urn contains m balls labeled from 1 to m. The experiment is to
choose a ball at random and record the number. Let X denote the
number. Then the distribution of X is given by
1
P (X = x) = , x = 1, . . . , m.
m
We do not know m and want to estimate m. So, θ
= m. We take a
sample of n balls denoted by X = {X1 , . . . , Xm }.
S OME E LEMENTARY S TATISTICAL I NFERENCES
• Sampling with Replacement: Here a ball is selected at random, its
number is recorded, the ball is replaced in the urn, the balls are then
remixed.
• Sampling without Replacement: Here a ball is selected at random.
If the balls are selected one-at-a-time, they are not replaced after each
draw.
• When m is much larger that n, the sampling schemes are practically
the same.
S OME E LEMENTARY S TATISTICAL I NFERENCES
• To estimate m we draw X1 , . . . , Xn with replacement. The
distribution of Xi is P (Xi = x) = 1/m for x = 1, . . . , m. An
intuitive estimator of m is the statistics T = max{X1 , . . . , Xn }.
How far is T from m? Prove that T is consistent estimator of m,
T →P m.
• In the above problem, E(X) = (m + 1)/2. Hence,
E(2X̄ − 1) = m. Perhaps, 2X̄ − 1 is also a good estimator of m
as it is unbiased. We will show that T is a better estimator in this case.
S OME E LEMENTARY S TATISTICAL I NFERENCES
An Example
• Suppose X is a random variable with unknown mean θ. Let
X1 , . . . , Xn be a random sample from the distribution of X and let
X̄ be the sample mean. Since E(X̄) = θ, X̄ is a point estimator of
θ. But how far is X̄ to θ? We will study the general case later. Now for
a specific case where X ∼ N (θ, σ 2 ). So, X̄ ∼ N (θ, σ 2 /n).
√
(X̄ − θ)/(σ/ n) has a standard normal distribution. So,
X̄ − θ
√ < 2)
0.954 = P (−2 <
σ/ n
σ
σ
= P (X̄ − 2 √ < θ < X̄ + 2 √ )
n
n
It says that before the sample is drawn the probability that the random
S OME E LEMENTARY S TATISTICAL I NFERENCES
− 2 √σn , X̄ + 2 √σn ) traps θ is 0.954. After the sample is
drawn the realized interval has either traps θ or it has not. But because
interval (X̄
of the high probability of success, namely 0.954, before the sample is
drawn, we call this interval a 95.4% confidence interval for θ .