Chapter 2
Principles of Maximum Likelihood Estimation and The
Analysis of Censored Data
Chapter 2
Principles of Maximum Likelihood Estimation and The
Analysis of Censored Data
Objectives
• Motivation for the use of parametric likelihood as a tool for
data analysis and inference.
Part of the Iowa State University NSF/ILI project
• Principles of likelihood and how likelihood is related to the
probability of the observed data.
Beyond Traditional Statistical Methods
• Likelihood for a parametric models.
Copyright 1999 D. Cook, W. M. Duckworth, M. S. Kaiser,
W. Q. Meeker, and W. R. Stephenson.
Developed as part of NSF/ILI grant DUE9751644.
• The use of likelihood confidence intervals for model parameters and other quantities of interest.
• Censoring mechanisms that restrict one’s ability to observe
actual response values.
• Likelihood for samples containing right and left censored
observations.
January 22, 2003
11h 20min
2-1
2-2
Example: Time Between α-Particle Emissions of
Americium-241 (Berkson 1966)
Importance of Maximum Likelihood
• Versatile method for fitting statistical models to data.
• Use a parametric statistical model to describe a set of data
or a process that generated a set of data.
• Much more general than least squares. Allows, in a relatively simple, coherent fashion:
Berkson (1966) investigates the randomness of α-particle
emissions of Americium-241, which has a half-life of about
458 years.
Data: Interarrival times (units: 1/5000 seconds).
• n =10,220 observations.
Censoring and truncation.
• Data binned into intervals from 0 to 4000 time
units. Interval sizes ranging from 25 to 100 units. Additional interval for observed times exceeding 4,000 time
units.
Non-normal distributions.
Non-independent observations.
Nonlinear regression models.
Multiple sources of variability.
• Smaller samples analyzed here to illustrate sample size effect. We start the analysis with n =200.
Time-dependent covariates.
2-3
2-4
Histogram of the n = 200 Sample of α-Particle
Interarrival Time Data
Data for α-Particle Emissions of Americium-241
41
44
24
32
29
21
9
0
200
20
30
1609
2424
1770
1306
1213
1528
354
16
10220
Frequency
100
300
500
700
1000
2000
4000
∞
Random Sample of Times
n = 200
dj
10
0
100
300
500
700
1000
2000
4000
All Times
n = 10220
0
Interval Endpoint
lower
upper
tj
tj−1
40
Interarrival Times
Frequency of Occurrence
Time
0
1000
2000
3000
4000
Time
2-5
2-6
Exponential Distribution
Exponential Distributions PDF and CDF
• The exponential cumulative distribution function (cdf) is
Probability Density Function
Cumulative Distribution Function
1
1.0
t
, t > 0,
θ
where θ is the single parameter of the distribution (equal to
the first moment or mean, in this example).
Pr(T ≤ t) = F (t; θ) = 1 − exp −
0.8
0.6
• The exponential distribution probability density function (pdf)
is
1
t
dF (t; θ)
= exp − .
f (t; θ) =
dt
θ
θ
F(t) .5
f(t)
0.4
0.2
0
0.0
0
1
2
3
4
• The exponential distribution quantile function is
0
1
2
t
3
4
tp = −θ log(1 − p).
t
2-7
2-8
Examples of Exponential Distributions
Parameters and Functions of Parameters
Cumulative Distribution Function
Probability Density Function
In practical applications, interest often centers on quantities
that are functions of θ like
2.0
1
1.5
F(t) .5
f(t) 1.0
0.5
• Probability p = Pr(T ≤ t) = F (t; θ ) for a specified t.
0.0
0
0.0
1.0
2.0
3.0
0.0
1.0
2.0
t
3.0
t
• The p quantile of the distribution of T [value of tp such that
F (tp ; θ ) = p].
Hazard Function
2.0
1.5
h(t)
1.0
0.5
0.0
1.0
2.0
3.0
θ
γ
0.5
1.0
2.0
0
0
0
• The mean of T
E(T ) =
∞
−∞
tf (t)dt
t
2-9
2 - 10
Lognormal Distribution
Lognormal Distributions PDF and CDF
• The Lognormal cumulative distribution function (cdf) is
Probability Density Function
Cumulative Distribution Function
1
log(t) − µ
,
σ
t > 0,
where µ is the mean of the logarithms and σ is the standard
deviation of the logarithms.
1.5
1.0
• The Lognormal distribution probability density function (pdf)
is
dF (t; µ, σ)
log(t) − µ
1
f (t; µ, σ) =
= φnor
.
dt
σt
σ
F(t) .5
f(t)
F (t; µ, σ) = Φnor
0.5
Shaded Area = .2
0.0
0
0.5
1.0
t
1.5
0.5
1.0
• The Lognormal distribution quantile function is
1.5
tp = exp µ + σΦ−1
nor (p) .
t
2 - 11
2 - 12
Examples of Lognormal Distributions
Cumulative Distribution Function
Examples of Weibull Distributions
Probability Density Function
Cumulative Distribution Function
1
Probability Density Function
1
1.2
F(t) .5
f(t)
0.8
F(t) .5
f(t)
0.4
0
0.0
0.0
1.0
2.0
3.0
0
0.0
1.0
2.0
t
3.0
0.0
3
2
1
0
1.0
2.0
1.5
2.0
0.0
0.5
1.0
1.5
2.0
t
Hazard Function
4
0.0
1.0
t
Hazard Function
h(t)
0.5
t
2.5
2.0
1.5
1.0
0.5
0.0
3.0
σ
µ
0.3
0.5
0.8
0
0
0
4
3
h(t)
2
1
0
0.0
t
0.5
1.0
1.5
2.0
β
η
0.8
1.0
1.5
1
1
1
t
2 - 13
Exponential Probability Plot of the n = 200 Sample of
α-Particle Interarrival Time Data. The Plot also
Shows Approximate 95% Simultaneous Nonparametric
Confidence Bands.
.99
2 - 14
Parametric Likelihood
Probability of the Data
• Using the model Pr(T ≤ t) = F (t; θ ) for continuous T ,
the likelihood (probability) for a single observation in the
interval (ti−1, ti] is
Li (θ ; datai) = Pr(ti−1 < T ≤ ti ) = F (ti ; θ ) − F (ti−1 ; θ ).
.98
Can be generalized to allow for explanatory variables, multiple sources of variability, and other model features.
Probability
.95
• The total likelihood is the joint probability of the data. Assuming n independent observations
.9
.8
.7
L(θ ) = L(θ ; DATA) = C
.6
.5
n
Li(θ ; datai).
i=1
.3
.1
0
500
1000
1500
• Want to estimate θ . Find values of θ for which L(θ ) is
relatively large.
2000
1/5000 Seconds
2 - 15
Exponential Distribution and Likelihood
for Interval Data
for the n = 200 α-Particle Interarrival
R(θ) = L(θ)/L(θ)
Time Data. Vertical Lines Give an Approximate 95%
Likelihood-Based Confidence Interval for θ
Data: α-particle emissions of americium-241
1.2
<- estimate of mean using all n=10220 times
t
F (t; θ) = 1 − exp −
, t > 0.
θ
θ = E(T ), the mean time between arrivals.
Relative Likelihood
1.0
• The interval-data likelihood has the form
L(θ) =
n
i=1
Li(θ) =
8 d
j
F (tj ; θ) − F (tj−1 ; θ)
j=1
8 tj−1
tj
=
exp −
− exp −
θ
θ
j=1
dj
0.8
0.50
0.60
0.6
0.70
0.80
0.4
n=200 ->
0.90
0.2
0.95
0.99
0.0
where dj is the number of interarrival times in the jth interval (i.e., times between tj−1 and tj ).
2 - 17
Confidence Level
• The exponential distribution is
2 - 16
400
600
800
1000
1200
1600
θ
2 - 18
Exponential Probability Plot for the n = 200 Sample of
α-Particle Interarrival Time Data. The Plot also
Shows Parametric Exponential ML Estimate and 95%
Confidence Intervals for F (t).
Example. α-Particle Pseudo Data Constructed
with Constant Proportion within Each Bin
Interarrival Times
Frequency of Occurrence
Time
.98
Interval Endpoint
lower
upper
tj
tj−1
Probability
.95
0
100
300
500
700
1000
2000
4000
.9
.8
.7
Exponential Distribution ML Fit
Exponential Distribution
95% Pointwise Confidence Intervals
.6
.5
.4
.2
0
500
1000
1500
100
300
500
700
1000
2000
4000
∞
2000
Samples of Times
n=20000 n=2000 n=200
dj
3000
5000
3000
3000
2000
3000
1000
0000
20000
300
500
300
300
200
300
100
000
2000
n=20
30
50
30
30
20
30
10
0
200
3
5
3
3
2
3
1
0
20
1/5000 Seconds
2 - 19
2 - 20
for the n = 20, 200, and 2000
R(θ) = L(θ)/L(θ)
Pseudo Data. Vertical Lines Give Corresponding
Approximate 95% Likelihood-Based Confidence
Intervals
Example. α-Particle Random Samples
Interarrival Times
Frequency of Occurrence
Time
Interval Endpoint
lower
upper
tj
tj−1
1.2
<- estimate of mean using all n=20000 times
0.8
0.50
0.60
0.6
0.70
n=20 ->
0.80
0.4
n=200 ->
0
100
300
500
700
1000
2000
4000
0.90
<- n=2000
0.2
Confidence Level
Relative Likelihood
1.0
0.95
0.99
0.0
400
600
800
1000
1200
100
300
500
700
1000
2000
4000
∞
1600
All Times
n = 10220
Random Samples of Times
n = 2000 n = 200 n=20
dj
1609
2424
1770
1306
1213
1528
354
16
10220
292
494
332
236
261
308
73
4
2000
41
44
24
32
29
21
9
0
200
3
7
4
1
3
2
0
0
20
θ
2 - 21
for the n = 20, 200, and 2000 Samples
R(θ) = L(θ)/L(θ)
from the α-Particle Interarrival Time Data. Vertical
Lines Give Corresponding Approximate 95%
Likelihood-Based Confidence Intervals.
2 - 22
Likelihood as a Tool for Modeling/Inference
What can we do with the (log) likelihood?
L(θ ) = log[L(θ )] =
1.2
estimate of mean using all n=10220 times ->
n
Li(θ ).
i=1
0.8
n=2000
0.50
0.60
0.6
0.70
n=20
0.80
0.4
n=200
• Study the surface.
Confidence Level
Relative Likelihood
1.0
• Maximize with respect to θ (ML point estimates).
0.90
• Look at curvature at maximum (gives estimate of Fisher
information and asymptotic variance).
0.2
0.95
0.99
0.0
200
400
600
800
1000
θ
2 - 23
2 - 24
Likelihood Ratio
Likelihood as a Tool for Modeling/Inference (Cont.)
• Regions of high likelihood are credible; regions of low likelihood are not credible (suggests confidence regions for parameters).
• The likelihood ratio to test for H0 : θ = θ0 is
L(θ0)/L(θ).
• The log likelihood ratio statistic for a particular θ0 is
• If the length of θ is > 1 or 2 and interest centers on subset of θ (need to get rid of nuisance parameters), look at
“profiles”
(suggests confidence regions/intervals for parameter subsets).
X 2 = −2 log L(θ0)/L(θ)
• Reject H0 at the 5% level of significance if X 2 > χ2
.
(1−α;1)
• For example, using the n = 200 α-particle data,
• Calibrate confidence regions/intervals with χ2 or simulation
(or parametric bootstrap).
−2 log [L(650)/L(572.3)] = 2.94 < χ2
(.95;1) = 3.84
• Likelihood ratio confidence interval: the set of all values of
θ such that H0 at the α level of significance is not rejected.
• Use “reparameterization” to study functions of θ .
2 - 25
Large-Sample Approximate Theory for Likelihood
Ratios for a Scalar Parameter
2 - 26
Normal-Approximation Confidence Intervals for θ
• A 100(1 − α)% normal-approximation (or Wald) confidence
interval for θ is
• Relative likelihood for θ is
R(θ) =
L(θ)
L(θ)
[θ ,
.
• If evaluated at the true θ, then, asymptotically (in large
samples), −2 log[R(θ)] follows, a chisquare distribution with
1 degrees of freedom.
• An approximate 100(1 − α)% likelihood-based confidence
region for θ is the set of all values of θ such that
where seθ =
θ̃] = θ ± z(1−α/2)seθ.
−d2L(θ)/dθ 2
−1
.
is evaluated at θ
• Based on
Zθ =
θ − θ
∼
˙ NOR(0, 1)
seθ
• From the definition of NOR(0, 1) quantiles
−2 log[R(θ)] < χ2
(1−α;1)
Pr z(α/2) < Zθ ≤ z(1−α/2) ≈ 1 − α
or, equivalently, the set defined by
implies that
R(θ) > exp −χ2
(1−α;1)/2 .
Pr θ − z(1−α/2)seθ < θ ≤ θ + z(1−α/2)seθ ≈ 1 − α.
2 - 27
2 - 28
Normal-Approximation Confidence Intervals for θ
(contd.)
Comparisons for α-Particle Data
• A 100(1 − α)% normal-approximation (or Wald) confidence
interval for θ is
[θ ,
θ̃] = [θ/w,
θ × w]
This follows after transformwhere w = exp[z(1−α/2) seθ/θ].
ing (by exponentiation) the confidence interval
[log(θ),
±z
˜
log(θ)]
= log(θ)
(1−α/2)selog(θ)
All Times
n =10,220
ML Estimate θ
Standard Error se
θ
95% Confidence Intervals
for θ Based on
Likelihood
Zlog(
∼
˙ NOR(0, 1)
θ)
Z
∼
˙ NOR(0, 1)
θ
× 105
ML Estimate λ
which is based on
− log(θ)
log(θ)
Zlog(θ)
∼
˙ NOR(0, 1)
=
selog(θ)
is unrestricted in sign, generally Z
• Because log(θ)
is
log(θ)
5
Standard Error se
λ×10
95% Confidence Intervals
5
for λ × 10 Based on
Likelihood
Zlog(
∼
˙ NOR(0, 1)
λ)
Z
∼
˙ NOR(0, 1)
λ
Sample of Times
n = 200
n=20
596
572
440
6.1
42.7
101
[585, 608]
[585, 608]
[585, 608]
[498, 662]
[496, 660]
[491, 654]
[289, 713]
[281, 690]
[242, 638]
168
175
227
1.7
13
52
[164, 171]
[164, 171]
[164, 171]
[151, 201]
[152, 202]
[149, 200]
[140, 346]
[145, 356]
[125, 329]
closer to an NOR(0, 1) distribution than is Zθ.
2 - 29
2 - 30
Confidence Intervals for Functions of θ
Leukemia Patient Remission Times
(from Lawless (1982)
• For one-parameter distributions, confidence intervals for
θ can be translated directly into confidence intervals for
monotone functions of θ.
• The arrival rate λ = 1/θ is a decreasing function of θ.
[λ,
λ̃] = [1/θ̃,
1/θ] = [.00151,
• The times marked with a * indicate patients that were still
in remission at the time the data were analyzed (rightcensored observations).
.00201].
• F (t; θ) is a decreasing function of θ
[F (te ),
F̃ (te )] = [F (te ; θ̃),
• At the time the data were analyzed
F (te ; θ )]
Twenty-five patients had come out of remission.
for any specified value of te.
• Quantile tp = −θ log(1 − p) is a increasing function of θ
t̃p] = [−θ log(1 − p),
[t ,
p
Five patients were still in remission.
• Analysts wanted to estimate the cdf of the remission-time
distribution.
−θ̃ log(1 − p)].
• The observed times were 1, 1, 2, 2, 2, 6, 6, 6, 7, 8, 9, 9,
10, 12, 13, 14, 18, 19 24, 26, 29, 31∗, 42, 45∗, 50∗, 57,
60, 71∗, 85∗, 91 weeks.
2 - 31
2 - 32
Density Approximation for Exact Observations
Exponential Probability Plot of the Remission Times
with 95% Simultaneous Confidence Bands.
• If ti−1 = ti − ∆i, ∆i > 0, and the correct likelihood
F (ti ; θ ) − F (ti−1 ; θ ) = F (ti ; θ ) − F (ti − ∆i; θ )
.98
can be approximated with the density f (t) as
Proportion Failing
.95
[F (ti ; θ ) − F (ti − ∆i; θ )] =
.9
t
i
(ti−∆i )
f (t)d t ≈ f (ti ; θ )∆i
then the density approximation for exact observations
Li(θ ; datai) = f (ti ; θ )
.8
.7
may be appropriate.
.6
.5
.4
• For most common models, the density approximation is adequate for small ∆i .
.2
.001
0
20
40
60
80
100
• There are, however, situations where the approximation
breaks down as ∆i → 0.
Weeks
2 - 33
Likelihood with Exact and Right-Censored
Observations
ML Estimates for the Exponential Distribution Mean
Based on the Density Approximation
• The probability of a right-censored the observation is
Li (θ ) = Pr(T > ti ) = F (∞; θ ) − F (ti ; θ ) = 1 − F (ti ; θ ).
• With n independent exact and right-censored observations,
the likelihood is:
L(θ ) =
n
{f (ti ; θ )}δi {1 − F (ti ; θ )}1−δi ,
where δi = 1 for an “exact” observation and δi = 0 for a
right-censored observation.
• For the exponential distribution,
n 1
t
exp − i
θ
θ
i=1
δi t
exp − i
θ
1−δ
i
• With r exact failures and n − r right-censored observations
the ML estimate of θ is
n
ti
TTT
= i=1
r
r
TTT = n
i=1 ti , total time on test, is the sum of the failure
times plus the censoring time of the units that are censored.
θ =
• Using the observed curvature in the log likelihood:
i=1
L(θ) =
2 - 34
−1
θ2
θ
d2L(θ) =√ .
seθ =
−
=
dθ 2 θ
r
r
• If the data are complete or failure censored, 2TTT /θ ∼ χ2
2r .
Then an exact 100(1 − α)% confidence interval for θ is
.

• It is easy to show that the value of θ that maximizes L(θ) is
n
θ = n
i=1 ti /r, where r = i=1 δi is the number of failures.
2 - 35
[θ ,
2(TTT )
θ̃] =  2
,
χ(1−α/2;2r)

2(TTT ) 
.
χ2
(α/2;2r)
2 - 36
Exponential Probability Plot of Remission Times with
Maximum Likelihood Estimates and 95% Pointwise
Confidence Intervals for F (t; θ)
Confidence Interval for the Mean Time in Remission
• The “total time on test” for the 30 patients is
TTT = 1 + 1 + 2 + 2 + 2 + 6 · · · + 85 + 91 = 756 weeks.
r = 25 patients came out of remission.
.99
.98
• The ML estimate of θ is
Proportion Failing
θ = 756/25 = 30.24 weeks.
• An approximate 95% confidence interval for θ is
θ , θ̃


2(756)
1901.76 1901.76
2(756) 
,
=  2
,
=
46.98
16.79
χ(.975;50) χ2
(.025;50)
.95
.9
.8
.7
.6
.4
= [21.17,
.2
.01
46.73]
(approximation due to the random censoring mechanism).
0
20
40
60
80
100
Weeks
2 - 37
2 - 38
Likelihood with Exact and Right-Censored
Observations
Lognormal Relative Likelihood Plot
for the Remission Times
• The probability of a right-censored the observation is
Li (θ ) = Pr(T ≤ ti ) = F (∞; θ ) − F (ti ; θ ) = 1 − F (ti ; θ ).
2.8
2.6
• With exact and right-censored observations, the likelihood
is:
δi
{f (ti ; θ )} {1 − F (ti ; θ )}
1−δi
2.2
2.0
Scale
L(θ ) =
n
0.001
2.4
,
1.8
i=1
0.1
1.6
where δi = 1 for an “exact” observation and δi = 0 for a
right-censored observation.
0.4 0.2
0.7
0.9
1.4
1.2
• For the lognormal distribution, θ = (µ, σ) and
L(µ, σ) =
n 1
σt
i=1
φnor
log(ti ) − µ
σ
δ i
1 − Φnor
0.001 0.01
1.0
log(ti ) − µ
σ
1−δ
0
i
0.001
10
20
30
40
0.5 Quantile
.
2 - 39
2 - 40
Lognormal Probability Plot of Remission Times with
Maximum Likelihood Estimates and Approximate 95%
Pointwise Confidence Intervals for F (t; µ, σ)
Lognormal Joint Confidence Region
for the Remission Times
.98
2.8
.95
2.6
.9
2.4
.8
Proportion Failing
Scale
2.2
2.0
1.8
90
95
60 70 80
25 50
1.6
.7
.6
.5
.4
.3
.2
1.4
.1
1.2
.05
.02
99
1.0
.01
0
10
20
30
.005
40
0.5 Quantile
1
2
5
10
20
50
100
Weeks
2 - 41
2 - 42
Lognormal Profile Likelihood R[exp(µ)] (exp(µ) = t.5)
for the Remission Time
Data
L(µ,σ)
R[exp(µ)] = max
L(µ
,σ
)
σ
Lognormal Profile Likelihood R(log(σ))
for the Remission Time Data
L(µ,log(σ))
R(log(σ)) = max
L(µ
,log(σ
))
µ
1.0
0.8
0.60
0.6
0.70
0.80
0.4
Profile Likelihood
0.50
Confidence Level
Profile Likelihood
0.8
0.50
0.60
0.6
0.70
0.80
0.4
0.90
Confidence Level
1.0
0.90
0.2
0.2
0.95
0.95
0.99
0.0
0
10
20
30
0.99
0.0
40
0.0
0.2
.50 Quantile (Median)
0.4
0.6
0.8
1.0
log sigma
2 - 43
Lognormal Profile Likelihood R(σ)
for the Remission Time Data
L(µ,σ)
R(σ) = max
L(µ
, σ
)
µ
2 - 44
Large-Sample Approximate Theory for Likelihood
Ratios for Parameter Vector Subset
Need: Inferences on subset θ 1, from the partition θ = (θ 1, θ 2).
• k1 = length(θ 1).
1.0
• When (θ 1, θ 2) = (µ, σ), profile likelihood for θ 1 = µ is
0.50
0.60
0.6
0.70
0.80
0.4
Confidence Level
Profile Likelihood
0.8
0.90
0.2
0.95
1.0
1.5
2.0
L(µ, σ)
.
σ
)
L(µ,
• If evaluated at the true θ 1 = µ, then, asymptotically, −2 log[R(µ)]
follows, a chisquare distribution with k1 = 1 degrees of freedom.
• General theory in the Appendix of Meeker and Escobar
(1998).
0.99
0.0
R(µ) = max
σ
2.5
σ
2 - 45
2 - 46
X-Ray to Optical Flux-Ratio Data
X-Ray Flux-Ratio Data
from a Sample of Active Galaxies
• Ratio of optical to X-ray flux for 107 active galaxies.
• For some galaxies, the flux values were too small to measure. This limitation in observation sensitivity causes left
censoring.
• Investigators were interested in quantifying the distribution
of this ratio among galaxies in a larger population, in order
to gain potential insight into possible physical causes for
the different types of radiation.
1.52
1.41
<1.82
1.56
1.25
1.42
1.44
1.22
<1.15
<1.47
1.87
<1.12
1.30
1.55
1.39
1.06
1.60
1.66
1.44
1.15
1.30
1.40
<1.18
1.54
1.59
1.42
1.22
1.44
1.50
1.00
1.58
1.15
1.67
1.14
1.30
<1.25
<1.40
1.55
1.52
1.65
1.30
1.59
<1.50
1.54
1.26
1.28
1.38
<1.38
1.51
<1.24
<1.33
<1.71
1.33
1.21
1.68
<1.57
1.68
1.48
1.34
1.43
<1.82
1.14
<1.17
1.38
1.71
1.38
1.46
1.61
1.21
1.48
<1.63
1.57
1.26
<1.38
1.25
<1.23
1.70
<1.32
1.45
1.66
0.91
1.38
<1.59
1.31
1.00
1.46
1.12
1.53
<1.56
1.43
1.48
<1.04
1.55
<1.36
<1.63
<1.68
1.47
1.46
1.22
1.71
1.62
<1.50
0.97
<1.57
1.21
1.36
1.18
Observations marked with a < are left censored.
2 - 47
2 - 48
Weibull probability plot of X-Ray Flux-Ratio Data with
95% Simultaneous Confidence Bands for F (t)
Lognormal probability plot of X-Ray Flux-Ratio Data
with 95% Simultaneous Confidence Bands for F (t)
.999
.9999
.98
.9995
.9
.998
.995
.5
.98
.3
.2
.95
Proportion Failing
Proportion Failing
.7
.1
.05
.03
.02
.01
.9
.8
.7
.5
.3
.2
.1
.005
.05
.003
.02
.001
.005
.002
.0005
.0005
0.8
1.0
1.2
1.4
1.6
1.8
2.0
0.8
1.0
1.2
Flux Ratio
1.4
1.6
1.8
2.0
Flux Ratio
2 - 49
2 - 50
Weibull Likelihood with
Exact and Left-Censored Observations
Relative Likelihood for the X-Ray Flux-Ratio Data and
the Lognormal Distribution
• The probability of a left-censored the observation is
Li(θ ) = Pr(−∞ < T ≤ ti ) = F (ti ; θ ) − F (−∞; θ ) = F (ti ; θ ).
0.22
0.20
0.001
• With exact and left-censored observations, the likelihood is:
n
{F (ti ; θ )}1−δi {f (ti ; θ )}δi ,
Scale
L(θ ) =
0.18
i=1
where δi = 1 for an “exact” observation and δi = 0 for a
left-censored observation.
0.14
0.12
• For the Weibull distribution,
n log(ti ) − µ
Φsev
L(µ, σ) =
σ
i=1
1−δ i
1
log(ti) − µ
φsev
σt
σ
0.01
0.1
0.7 0.4 0.2
0.9
0.16
0.10
δ
i
1.26
1.30
.
1.34
1.38
1.42
0.5 Quantile
2 - 51
2 - 52
Weibull Probability Plot of X-Ray Flux-Ratio Data
with Weibull and Lognormal Maximum Likelihood
Estimates and 95% Pointwise Confidence Intervals for
the Weibull F (t; µ, σ)
Lognormal Joint Confidence Region
for the X-Ray Flux-Ratio Data
0.22
.999
.98
0.20
.9
.7
0.16
25
50 70 80
60
90 95
99
Proportion Failing
Scale
0.18
0.14
.5
.3
.2
.1
•
0.12
.05
••
• •
•
Lognormal Distribution ML Fit
Weibull Distribution ML Fit
Lognormal Distribution 95% Pointwise Confidence Intervals
.03
0.10
.02
1.26
1.30
1.34
1.38
•
•
••
•• ••
•••••
•
•
•
•
•••••
••••
•••••
•
•
•
•••
••
••
•
1.42
.01
0.5 Quantile
0.8
1.0
1.2
1.4
1.6
1.8
2.0
Flux Ratio
2 - 53
2 - 54
Comparison Profile Likelihoods for the Median of the
X-Ray Flux-Ratio Distribution Using Weibull and
Lognormal Distributions
1.0
0.50
<-Weibull
lognormal->
0.60
0.6
0.70
0.80
0.4
Confidence Level
Profile Likelihood
0.8
0.90
0.2
0.95
0.99
0.0
1.28
1.30
1.32
1.34
1.36
1.38
1.40
1.42
.50 Quantile (Median)
2 - 55