EXAMPLES OF STATISTICAL LIKELIHOODS MAXIMUM LIKELIHOOD ESTIMATES 1. Binomial likelihood Suppose that you perform n independent success/fail trials in which the success probability is p at each trial. For i = 1, 2, …, n, let 0 Yi = 1 if trial i is a failure if trial i is a success Let likelihood for Yi is Li = pYi 1 p i . This is especially simple as Yi only has the values 0 and 1. This likelihood will be either 1 – p or p. 1Y n The likelihood for the whole problem is L = Li = i 1 p 1 p n 1 Yi Yi i 1 n Yi n Yi . p i 1 1 p i 1 n = n let d Use log L . Then do log L 0 and find easily pˆ ML dp y i 1 i . In random n n variable form, this is pˆ ML Y i 1 n i . 2. Normal sample likelihood Suppose that X1, X2, …, Xn is a sample from the normal distribution with mean and standard deviation . The likelihood is n 1 2 xi 2 xi 1 1 2 2 i 1 e L = Li = = e 2 n /2 n 2 i 1 i 1 2 n 1 n Use then log L = n log n 1 log 2 2 2 2 let d log L 0 d let d log L 0 d 1 n x i 1 i . 2 2 Solve the system EXAMPLES OF STATISTICAL LIKELIHOODS MAXIMUM LIKELIHOOD ESTIMATES The solution will be ˆ ML x 1 ˆ ML n n x i i 1 x 2 This does not have the usual n – 1 of the sample standard deviation s. 3. Regression likelihood Suppose that x1, x2, … , xn are fixed numbers. Suppose that Y1, Y2, …, Yn are independent random variables with Yi ~ N(0 + 1 xi , 2). The likelihood is n 2 1 2 yi 0 1 xi 2 yi 0 1 xi 1 1 2 i 1 2 e L = Li = = e n /2 n 2 i 1 i 1 2 n 1 n 2 The maximum likelihood estimates ̂ 0 and ̂1 are familiar. It can be shown that ˆ ML 1 n y n i 1 i ˆ 0 ˆ 1 xi . 2 4. Exponential sample Suppose that X1, X2, …, Xn is from the exponential density with mean . The likelihood is n 1 xi 1 xi L = Li = e = n e i 1 i 1 i 1 n n 1 The maximum likelihood estimate is ˆ ML x . If the density were written as e x , 1 the maximum likelihood estimate would be ˆ ML . x 5. Censored exponential sample Suppose that X1, X2, …, Xn are independent random variables, each from the density e -x. Actually we are able to observe the Xi’s only if they have a value T or below. 2 EXAMPLES OF STATISTICAL LIKELIHOODS MAXIMUM LIKELIHOOD ESTIMATES This corresponds to an experimental framework in which we are observing the lifetimes of n independent objects (light bulbs, say), but that the experiment ceases at time T. Suppose that K of the Xi’s are observed; call these values X 1 , X 2 , X 3 , .... , X K . The remaining n - K values are censored at T ; operationally, this means that there were n - K light bulbs still burning when the experiment stopped at time T. The overall likelihood is L = e T n K K e K xi i 1 It is not at all obvious what would be the maximum likelihood estimate. Let’s take logs: K log L = T n K K log xi i 1 Then find d K log L = T n K d This leads to ˆ ML = K T n K K x i 1 K let x i 1 . 0 i This is certainly interesting. i 6. Birth process Start with X0 = 1. Thereafter, let the conditional distribution of Xt | Xt – 1 = xt - 1 be Poisson with parameter xt – 1. The idea is that one female (for generation 0) produces a number of daughters which follows a Poisson distribution with parameter . Thereafter, these X1 daughters (generation 1) will independently produce daughters according to a Poisson distribution with parameter . The total number of daughters for generation 2 follows a Poisson distribution with parameter X1. Observe that if at any time the total number of daughters produced is zero, then the process dies out forever. 3 EXAMPLES OF STATISTICAL LIKELIHOODS MAXIMUM LIKELIHOOD ESTIMATES The likelihood through K generations is K L = e xi K 1 xi i 1 i 0 K x ! i 1 K 1 x xi 1 i i 1 i Then find K 1 K log L = xi xi log terms without i 0 i 1 We set up K K 1 d log L = xi d i 0 x i 1 i let 0 The solution is the surprising form K ˆ ML = x i 1 K 1 i x i 0 i 4