A geometric view of overdispersion
Karim Anaya-Izquierdo (The Open University)
joint work with Frank Critchley (Open University),
Paul Marriott (Waterloo) and Paul Vos (East Carolina)
WOGAS 3, Warwick 2011
A geometric treatment of overdispersion
Introduction
Overdispersion is common in practice
A geometric treatment of overdispersion
Introduction
Overdispersion is common in practice
Target: Overdispersed exponential families (Binomial)
A geometric treatment of overdispersion
Introduction
Overdispersion is common in practice
Target: Overdispersed exponential families (Binomial)
Models for overdispersed data can be expressed in terms of
simple geometrical operations
A geometric treatment of overdispersion
Overdispersion
Exponential family
f (y ; µ) = exp(θ (µ)y − ψ(θ (µ))) ν(y )
A geometric treatment of overdispersion
Overdispersion
Exponential family
f (y ; µ) = exp(θ (µ)y − ψ(θ (µ))) ν(y )
where E [Y ] = µ = ψ0 (θ (µ))
A geometric treatment of overdispersion
Overdispersion
Exponential family
f (y ; µ) = exp(θ (µ)y − ψ(θ (µ))) ν(y )
where E [Y ] = µ = ψ0 (θ (µ)) and ν(y ) is the base measure
A geometric treatment of overdispersion
Overdispersion
Exponential family
f (y ; µ) = exp(θ (µ)y − ψ(θ (µ))) ν(y )
where E [Y ] = µ = ψ0 (θ (µ)) and ν(y ) is the base measure
The variance function
Var (Y ) = ψ00 (θ (µ)) = v (µ)
so the variance depends on the mean
A geometric treatment of overdispersion
Overdispersion
Exponential family
f (y ; µ) = exp(θ (µ)y − ψ(θ (µ))) ν(y )
where E [Y ] = µ = ψ0 (θ (µ)) and ν(y ) is the base measure
The variance function
Var (Y ) = ψ00 (θ (µ)) = v (µ)
so the variance depends on the mean
Usually we observe in data that nominal variance is
larger/smaller than one expected by exponential family
A geometric treatment of overdispersion
Overdispersion
Exponential family
f (y ; µ) = exp(θ (µ)y − ψ(θ (µ))) ν(y )
where E [Y ] = µ = ψ0 (θ (µ)) and ν(y ) is the base measure
The variance function
Var (Y ) = ψ00 (θ (µ)) = v (µ)
so the variance depends on the mean
Usually we observe in data that nominal variance is
larger/smaller than one expected by exponential family
We can proceed depending of type/cause of overdispersion by
modelling a more flexible variance function eg by adding extra
parameters
A geometric treatment of overdispersion
Overdispersion
Need a distribution where a single parameter controls
dispersion
A geometric treatment of overdispersion
Overdispersion
Need a distribution where a single parameter controls
dispersion
Prototypical example:
A geometric treatment of overdispersion
Overdispersion
Need a distribution where a single parameter controls
dispersion
Prototypical example: Normal distribution N (µ, σ2 )!
A geometric treatment of overdispersion
Overdispersion
Need a distribution where a single parameter controls
dispersion
Prototypical example: Normal distribution N (µ, σ2 )!
Generally: Exponential dispersion models
g (y ; θ, ψ) = exp((y θ − ψ(θ ))/φ) νφ (y )
A geometric treatment of overdispersion
Overdispersion
Need a distribution where a single parameter controls
dispersion
Prototypical example: Normal distribution N (µ, σ2 )!
Generally: Exponential dispersion models
g (y ; θ, ψ) = exp((y θ − ψ(θ ))/φ) νφ (y )
Eg [ Y ] = µ = ψ 0 ( θ )
Varg [Y ] = φ ψ00 (θ (µ)) = φ v (µ)
A geometric treatment of overdispersion
Overdispersion
Need a distribution where a single parameter controls
dispersion
Prototypical example: Normal distribution N (µ, σ2 )!
Generally: Exponential dispersion models
g (y ; θ, ψ) = exp((y θ − ψ(θ ))/φ) νφ (y )
Eg [ Y ] = µ = ψ 0 ( θ )
Varg [Y ] = φ ψ00 (θ (µ)) = φ v (µ)
Do not exist for discrete data! Jorgensen (1997)
A geometric treatment of overdispersion
Binomial Y ∼ Bin (n, µ)
f (y ; µ, n) =
n −n y
n µ ( n − µ ) n −y ,
x
ν (y ) =
n
y
Ef [ Y ] = µ
µ (n − µ )
= v (µ)
Varf [Y ] =
n
A geometric treatment of overdispersion
Binomial Y ∼ Bin (n, µ)
f (y ; µ, n) =
n −n y
n µ ( n − µ ) n −y ,
x
ν (y ) =
n
y
Ef [ Y ] = µ
µ (n − µ )
= v (µ)
Varf [Y ] =
n
Overdispersion by mixing:
A geometric treatment of overdispersion
Binomial Y ∼ Bin (n, µ)
f (y ; µ, n) =
n −n y
n µ ( n − µ ) n −y ,
x
ν (y ) =
n
y
Ef [ Y ] = µ
µ (n − µ )
= v (µ)
Varf [Y ] =
n
Overdispersion by mixing:
Y | M ∼ Bin(n, M )
A geometric treatment of overdispersion
Binomial Y ∼ Bin (n, µ)
f (y ; µ, n) =
n −n y
n µ ( n − µ ) n −y ,
x
ν (y ) =
n
y
Ef [ Y ] = µ
µ (n − µ )
= v (µ)
Varf [Y ] =
n
Overdispersion by mixing:
M∼H
such that
Y | M ∼ Bin(n, M )
EH [ M ] = µ
VarH [M ] = τ 2 µ (n − µ)
A geometric treatment of overdispersion
Binomial Y ∼ Bin (n, µ)
f (y ; µ, n) =
n −n y
n µ ( n − µ ) n −y ,
x
ν (y ) =
n
y
Ef [ Y ] = µ
µ (n − µ )
= v (µ)
Varf [Y ] =
n
Overdispersion by mixing:
M∼H
such that
Y | M ∼ Bin(n, M )
EH [ M ] = µ
VarH [M ] = τ 2 µ (n − µ)
marginally
A geometric treatment of overdispersion
Binomial Y ∼ Bin (n, µ)
f (y ; µ, n) =
n −n y
n µ ( n − µ ) n −y ,
x
ν (y ) =
n
y
Ef [ Y ] = µ
µ (n − µ )
= v (µ)
Varf [Y ] =
n
Y | M ∼ Bin(n, M )
Overdispersion by mixing:
M∼H
such that
EH [ M ] = µ
marginally g (x; µ, τ 2 ) =
Z n
VarH [M ] = τ 2 µ (n − µ)
f (x; m, n)dH (m )
0
A geometric treatment of overdispersion
Binomial Y ∼ Bin (n, µ)
f (y ; µ, n) =
n −n y
n µ ( n − µ ) n −y ,
x
ν (y ) =
n
y
Ef [ Y ] = µ
µ (n − µ )
= v (µ)
Varf [Y ] =
n
Y | M ∼ Bin(n, M )
Overdispersion by mixing:
M∼H
such that
EH [ M ] = µ
marginally g (x; µ, τ 2 ) =
Z n
VarH [M ] = τ 2 µ (n − µ)
f (x; m, n)dH (m )
0
Eg [ Y ] = µ
Varg [Y ] = v (µ)[1 + (n − 1)τ 2 ]
A geometric treatment of overdispersion
Binomial Y ∼ Bin (n, µ)
f (y ; µ, n) =
n −n y
n µ ( n − µ ) n −y ,
x
ν (y ) =
n
y
Ef [ Y ] = µ
µ (n − µ )
= v (µ)
Varf [Y ] =
n
Y | M ∼ Bin(n, M )
Overdispersion by mixing:
M∼H
such that
EH [ M ] = µ
marginally g (x; µ, τ 2 ) =
Z n
VarH [M ] = τ 2 µ (n − µ)
f (x; m, n)dH (m )
0
Eg [ Y ] = µ
Varg [Y ] = v (µ)[1 + (n − 1)τ 2 ]
Example: Beta-Binomial
A geometric treatment of overdispersion
Binomial
Extension
f (y ; µ) → g (y ; µ, φ)
A geometric treatment of overdispersion
Binomial
Extension
f (y ; µ) → g (y ; µ, φ)
so that
g (y ; µ, 0) = f (y ; µ)
A geometric treatment of overdispersion
Binomial
Extension
f (y ; µ) → g (y ; µ, φ)
so that
g (y ; µ, 0) = f (y ; µ)
E f [ Y ] = Eg [ Y ] = µ
A geometric treatment of overdispersion
Binomial
Extension
f (y ; µ) → g (y ; µ, φ)
so that
g (y ; µ, 0) = f (y ; µ)
E f [ Y ] = Eg [ Y ] = µ
Varg [Y ] ≥ Varf [Y ] = v (µ)
∀µ
A geometric treatment of overdispersion
Binomial
Extension
f (y ; µ) → g (y ; µ, φ)
so that
g (y ; µ, 0) = f (y ; µ)
E f [ Y ] = Eg [ Y ] = µ
Varg [Y ] ≥ Varf [Y ] = v (µ)
∀µ
How do we extend in general?
A geometric treatment of overdispersion
Binomial
Extension
f (y ; µ) → g (y ; µ, φ)
so that
g (y ; µ, 0) = f (y ; µ)
E f [ Y ] = Eg [ Y ] = µ
Varg [Y ] ≥ Varf [Y ] = v (µ)
∀µ
How do we extend in general?
Flexibility
A geometric treatment of overdispersion
Binomial
Extension
f (y ; µ) → g (y ; µ, φ)
so that
g (y ; µ, 0) = f (y ; µ)
E f [ Y ] = Eg [ Y ] = µ
Varg [Y ] ≥ Varf [Y ] = v (µ)
∀µ
How do we extend in general?
Flexibility
Parsimony
A geometric treatment of overdispersion
Binomial
Extension
f (y ; µ) → g (y ; µ, φ)
so that
g (y ; µ, 0) = f (y ; µ)
E f [ Y ] = Eg [ Y ] = µ
Varg [Y ] ≥ Varf [Y ] = v (µ)
∀µ
How do we extend in general?
Flexibility
Parsimony
Use simple dual affine geometry!!
A geometric treatment of overdispersion
Binomial
Extension
f (y ; µ) → g (y ; µ, φ)
so that
g (y ; µ, 0) = f (y ; µ)
E f [ Y ] = Eg [ Y ] = µ
Varg [Y ] ≥ Varf [Y ] = v (µ)
∀µ
How do we extend in general?
Flexibility
Parsimony
Use simple dual affine geometry!! -1 type and -1 type extensions
A geometric treatment of overdispersion
Overdispersed Binomials: +1 type extensions
f (y ; µ) = exp(y θ (µ) − ψ(θ (µ))) ν(y )
↓
g (y ; µ, φ) = exp(y η + T (y )φ − κ (η, φ)) ν(y )
where µ = Eg [Y ].
A geometric treatment of overdispersion
Overdispersed Binomials: +1 type extensions
f (y ; µ) = exp(y θ (µ) − ψ(θ (µ))) ν(y )
↓
g (y ; µ, φ) = exp(y η + T (y )φ − κ (η, φ)) ν(y )
where µ = Eg [Y ]. Examples
A geometric treatment of overdispersion
Overdispersed Binomials: +1 type extensions
f (y ; µ) = exp(y θ (µ) − ψ(θ (µ))) ν(y )
↓
g (y ; µ, φ) = exp(y η + T (y )φ − κ (η, φ)) ν(y )
where µ = Eg [Y ]. Examples
Family
T (y )
Double Binomial
y log(y /n) + (n − y ) log((n − y )/n)
Multiplicative
−x (n − x )
Shrink base measure
− log (xn)
A geometric treatment of overdispersion
Overdispersed Binomials: +1 type extensions
Locally to φ = 0
Var (Y ; µ, η ) = v (µ) + φ ξ (µ)
where
ξ (µ) = covf (T (Y ), Y 2 ) −
covf (Y , Y 2 )covf (T (Y ), Y )
Varf (Y )
A geometric treatment of overdispersion
Overdispersed Binomials: +1 type extensions
Locally to φ = 0
Var (Y ; µ, η ) = v (µ) + φ ξ (µ)
where
covf (Y , Y 2 )covf (T (Y ), Y )
Varf (Y )
If T(y) convex then overdispersion (Gelfand & Dalal 1990,
Lindsay, 1986)
ξ (µ) = covf (T (Y ), Y 2 ) −
A geometric treatment of overdispersion
Overdispersed Binomials: +1 type extensions
Locally to φ = 0
Var (Y ; µ, η ) = v (µ) + φ ξ (µ)
where
covf (Y , Y 2 )covf (T (Y ), Y )
Varf (Y )
If T(y) convex then overdispersion (Gelfand & Dalal 1990,
Lindsay, 1986)
ξ (µ) = covf (T (Y ), Y 2 ) −
Parameters η and µ = Eg (Y ) are orthogonal (mixed
parametrisation)
A geometric treatment of overdispersion
Double Binomial (Efron, 1986)
Consider the +1 joining of f (y ; µ) and f (y ; y )
g (y ; µ, λ) = c (µ, λ)[f (y ; µ)]λ [f (y ; y )]1−λ
A geometric treatment of overdispersion
Double Binomial (Efron, 1986)
Consider the +1 joining of f (y ; µ) and f (y ; y )
g (y ; µ, λ) = c (µ, λ)[f (y ; µ)]λ [f (y ; y )]1−λ
which can be written as
g (y ; µ, φ) = exp(y η + T (y )φ − κ (η, φ))
where
φ = 1−λ
η = λ θ (µ)
A geometric treatment of overdispersion
Double Binomial (Efron, 1986)
Consider the +1 joining of f (y ; µ) and f (y ; y )
g (y ; µ, λ) = c (µ, λ)[f (y ; µ)]λ [f (y ; y )]1−λ
which can be written as
g (y ; µ, φ) = exp(y η + T (y )φ − κ (η, φ))
where
φ = 1−λ
η = λ θ (µ)
Interpretation
A geometric treatment of overdispersion
Double Binomial (Efron, 1986)
Consider the +1 joining of f (y ; µ) and f (y ; y )
g (y ; µ, λ) = c (µ, λ)[f (y ; µ)]λ [f (y ; y )]1−λ
which can be written as
g (y ; µ, φ) = exp(y η + T (y )φ − κ (η, φ))
where
φ = 1−λ
η = λ θ (µ)
Interpretation
φ > 0 overdispersion
A geometric treatment of overdispersion
Double Binomial (Efron, 1986)
Consider the +1 joining of f (y ; µ) and f (y ; y )
g (y ; µ, λ) = c (µ, λ)[f (y ; µ)]λ [f (y ; y )]1−λ
which can be written as
g (y ; µ, φ) = exp(y η + T (y )φ − κ (η, φ))
where
φ = 1−λ
η = λ θ (µ)
Interpretation
φ > 0 overdispersion
φ < 0 underdispersion
A geometric treatment of overdispersion
Shrinking base measure, (Shmueli, et al 2005)
Consider
λ
n
g (y ; µ, λ) =
exp(y η − κ (λ, η ))
y
n
=
exp(y η + φT (y ) − κ (η, φ))
y
A geometric treatment of overdispersion
Shrinking base measure, (Shmueli, et al 2005)
Consider
λ
n
g (y ; µ, λ) =
exp(y η − κ (λ, η ))
y
n
=
exp(y η + φT (y ) − κ (η, φ))
y
where φ = 1 − λ < 1 and T (y ) = − log (yn )
A geometric treatment of overdispersion
Shrinking base measure, (Shmueli, et al 2005)
Consider
λ
n
g (y ; µ, λ) =
exp(y η − κ (λ, η ))
y
n
=
exp(y η + φT (y ) − κ (η, φ))
y
where φ = 1 − λ < 1 and T (y ) = − log (yn )
Interpretation
A geometric treatment of overdispersion
Shrinking base measure, (Shmueli, et al 2005)
Consider
λ
n
g (y ; µ, λ) =
exp(y η − κ (λ, η ))
y
n
=
exp(y η + φT (y ) − κ (η, φ))
y
where φ = 1 − λ < 1 and T (y ) = − log (yn )
Interpretation
φ > 0 (λ < 1) overdispersion
A geometric treatment of overdispersion
Shrinking base measure, (Shmueli, et al 2005)
Consider
λ
n
g (y ; µ, λ) =
exp(y η − κ (λ, η ))
y
n
=
exp(y η + φT (y ) − κ (η, φ))
y
where φ = 1 − λ < 1 and T (y ) = − log (yn )
Interpretation
φ > 0 (λ < 1) overdispersion
φ < 0 (λ > 1) underdispersion
A geometric treatment of overdispersion
Multiplicative Binomial (Altham, 1978)
Consider
g (y ; µ, φ) =
n
exp(ηy + φT (y ) − κ (η, φ))
y
where T (y ) = −y (n − y )
A geometric treatment of overdispersion
Multiplicative Binomial (Altham, 1978)
Consider
g (y ; µ, φ) =
n
exp(ηy + φT (y ) − κ (η, φ))
y
where T (y ) = −y (n − y )
Distribution of the sum of n dependent binary variables with
symmetric joint distribution and no second or higher order
multiplicative interactions
A geometric treatment of overdispersion
Multiplicative Binomial (Altham, 1978)
Consider
g (y ; µ, φ) =
n
exp(ηy + φT (y ) − κ (η, φ))
y
where T (y ) = −y (n − y )
Distribution of the sum of n dependent binary variables with
symmetric joint distribution and no second or higher order
multiplicative interactions
Var [Y ] = v (µ) + (m − 1)r (µ, φ)
A geometric treatment of overdispersion
Multiplicative Binomial (Altham, 1978)
Consider
g (y ; µ, φ) =
n
exp(ηy + φT (y ) − κ (η, φ))
y
where T (y ) = −y (n − y )
Distribution of the sum of n dependent binary variables with
symmetric joint distribution and no second or higher order
multiplicative interactions
Var [Y ] = v (µ) + (m − 1)r (µ, φ)
Interpretation
A geometric treatment of overdispersion
Multiplicative Binomial (Altham, 1978)
Consider
g (y ; µ, φ) =
n
exp(ηy + φT (y ) − κ (η, φ))
y
where T (y ) = −y (n − y )
Distribution of the sum of n dependent binary variables with
symmetric joint distribution and no second or higher order
multiplicative interactions
Var [Y ] = v (µ) + (m − 1)r (µ, φ)
Interpretation
φ > 0 (positive association) overdispersion
A geometric treatment of overdispersion
Multiplicative Binomial (Altham, 1978)
Consider
g (y ; µ, φ) =
n
exp(ηy + φT (y ) − κ (η, φ))
y
where T (y ) = −y (n − y )
Distribution of the sum of n dependent binary variables with
symmetric joint distribution and no second or higher order
multiplicative interactions
Var [Y ] = v (µ) + (m − 1)r (µ, φ)
Interpretation
φ > 0 (positive association) overdispersion
φ < 0 (negative association) underdispersion
A geometric treatment of overdispersion
Overdispersed Binomials: +1 type extension
Let now T (y ) to depend on µ
f (y ; µ) = exp(y θ (µ) − ψ(θ (µ))) ν(y )
↓
g (y ; µ, φ) = exp(y η + T (y , µ)φ − κ (η, φ)) ν(y )
A geometric treatment of overdispersion
Overdispersed Binomials: +1 type extension
Let now T (y ) to depend on µ
f (y ; µ) = exp(y θ (µ) − ψ(θ (µ))) ν(y )
↓
g (y ; µ, φ) = exp(y η + T (y , µ)φ − κ (η, φ)) ν(y )
Examples
T (y )
Family
Overdispersion Test
Copas & Eguchi (2005)
d 2f
(x; θ (µ))/d µ2
f (x; θ (µ))
semiparametric
A geometric treatment of overdispersion
Overdispersion test (Cox, 1983, Rayner & Best, 2009)
g (y ; µ, φ) with
T (y ; µ ) =
d 2 f (x; θ (µ))/d µ2
f (x; θ (µ))
is a local mixture in the sense
A geometric treatment of overdispersion
Overdispersion test (Cox, 1983, Rayner & Best, 2009)
g (y ; µ, φ) with
T (y ; µ ) =
d 2 f (x; θ (µ))/d µ2
f (x; θ (µ))
is a local mixture in the sense
Z
f (x; m )dH (m ) ≈ g (x; µ, φ)
for small mixing distribution H s.t EH [M ] = µ
A geometric treatment of overdispersion
Overdispersion test (Cox, 1983, Rayner & Best, 2009)
g (y ; µ, φ) with
T (y ; µ ) =
d 2 f (x; θ (µ))/d µ2
f (x; θ (µ))
is a local mixture in the sense
Z
f (x; m )dH (m ) ≈ g (x; µ, φ)
for small mixing distribution H s.t EH [M ] = µ
Neyman’s smooth goodness of fit tests. Locally most powerful
directed tests .
A geometric treatment of overdispersion
Overdispersion test (Cox, 1983, Rayner & Best, 2009)
g (y ; µ, φ) with
T (y ; µ ) =
d 2 f (x; θ (µ))/d µ2
f (x; θ (µ))
is a local mixture in the sense
Z
f (x; m )dH (m ) ≈ g (x; µ, φ)
for small mixing distribution H s.t EH [M ] = µ
Neyman’s smooth goodness of fit tests. Locally most powerful
directed tests .
Interpretation
A geometric treatment of overdispersion
Overdispersion test (Cox, 1983, Rayner & Best, 2009)
g (y ; µ, φ) with
T (y ; µ ) =
d 2 f (x; θ (µ))/d µ2
f (x; θ (µ))
is a local mixture in the sense
Z
f (x; m )dH (m ) ≈ g (x; µ, φ)
for small mixing distribution H s.t EH [M ] = µ
Neyman’s smooth goodness of fit tests. Locally most powerful
directed tests .
Interpretation
φ > 0 overdispersion
A geometric treatment of overdispersion
Overdispersion test (Cox, 1983, Rayner & Best, 2009)
g (y ; µ, φ) with
T (y ; µ ) =
d 2 f (x; θ (µ))/d µ2
f (x; θ (µ))
is a local mixture in the sense
Z
f (x; m )dH (m ) ≈ g (x; µ, φ)
for small mixing distribution H s.t EH [M ] = µ
Neyman’s smooth goodness of fit tests. Locally most powerful
directed tests .
Interpretation
φ > 0 overdispersion
φ < 0 underspersion
A geometric treatment of overdispersion
Overdispersed Binomials: -1 type extensions
A different approach is to use local-mixture-like expressions such as
f (y ; µ) = exp(y θ (µ) − ψ(θ (µ))) ν(y )
↓
g (y ; µ, φ) = f (y ; µ)[1 + φ T (y , θ )]
A geometric treatment of overdispersion
Overdispersed Binomials: -1 type extensions
A different approach is to use local-mixture-like expressions such as
f (y ; µ) = exp(y θ (µ) − ψ(θ (µ))) ν(y )
↓
g (y ; µ, φ) = f (y ; µ)[1 + φ T (y , θ )]
Examples
T (y )
Family
Local Mixture order 2
Correlated Binomial
d 2f
(x; θ (µ))/d µ2
f (x; θ (µ))
ditto
A geometric treatment of overdispersion
Binomial Local mixture: Anaya-Izquierdo & Marriott(2007)
Modified Laplace expansion
Z
d 2 f (x; θ (µ))/d µ2
f (y ; m ) dH (m ) ∼ g (y ; µ, φ) = f (x; µ) 1 + φ
f (x; θ (µ))
A geometric treatment of overdispersion
Binomial Local mixture: Anaya-Izquierdo & Marriott(2007)
Modified Laplace expansion
Z
d 2 f (x; θ (µ))/d µ2
f (y ; m ) dH (m ) ∼ g (y ; µ, φ) = f (x; µ) 1 + φ
f (x; θ (µ))
var [Y ] = v (µ) +
(n −1)
n φ
A geometric treatment of overdispersion
Binomial Local mixture: Anaya-Izquierdo & Marriott(2007)
Modified Laplace expansion
Z
d 2 f (x; θ (µ))/d µ2
f (y ; m ) dH (m ) ∼ g (y ; µ, φ) = f (x; µ) 1 + φ
f (x; θ (µ))
var [Y ] = v (µ) +
(n −1)
n φ
µ and φ are orthogonal
A geometric treatment of overdispersion
Binomial Local mixture: Anaya-Izquierdo & Marriott(2007)
Modified Laplace expansion
Z
d 2 f (x; θ (µ))/d µ2
f (y ; m ) dH (m ) ∼ g (y ; µ, φ) = f (x; µ) 1 + φ
f (x; θ (µ))
var [Y ] = v (µ) +
(n −1)
n φ
µ and φ are orthogonal
Interpretation
A geometric treatment of overdispersion
Binomial Local mixture: Anaya-Izquierdo & Marriott(2007)
Modified Laplace expansion
Z
d 2 f (x; θ (µ))/d µ2
f (y ; m ) dH (m ) ∼ g (y ; µ, φ) = f (x; µ) 1 + φ
f (x; θ (µ))
var [Y ] = v (µ) +
(n −1)
n φ
µ and φ are orthogonal
Interpretation
φ > 0 overdispersion
A geometric treatment of overdispersion
Binomial Local mixture: Anaya-Izquierdo & Marriott(2007)
Modified Laplace expansion
Z
d 2 f (x; θ (µ))/d µ2
f (y ; m ) dH (m ) ∼ g (y ; µ, φ) = f (x; µ) 1 + φ
f (x; θ (µ))
var [Y ] = v (µ) +
(n −1)
n φ
µ and φ are orthogonal
Interpretation
φ > 0 overdispersion
φ < 0 underdispersion
A geometric treatment of overdispersion
Correlated Binomial (Kupper & Haseman, 1978)
Distribution of the sum of n dependent binary variables where
their joint distribution is symmetric with no second or higher
order additive interactions
A geometric treatment of overdispersion
Correlated Binomial (Kupper & Haseman, 1978)
Distribution of the sum of n dependent binary variables where
their joint distribution is symmetric with no second or higher
order additive interactions
Take ρ = φ/(µ(n − µ)) in Local mixture so ρ has the
interpretation of being a pairwise correlation between the
binary variables and
Var [Y ] = v (µ)[1 + (n − 1)ρ]
A geometric treatment of overdispersion
Correlated Binomial (Kupper & Haseman, 1978)
Distribution of the sum of n dependent binary variables where
their joint distribution is symmetric with no second or higher
order additive interactions
Take ρ = φ/(µ(n − µ)) in Local mixture so ρ has the
interpretation of being a pairwise correlation between the
binary variables and
Var [Y ] = v (µ)[1 + (n − 1)ρ]
In general, let Z1 , . . . , Zn be dependent Bernoulli(p ) random
variables such that Cov [Zi , Zj ] = φ for i 6= j. If Y = ∑ni=1 Zi
n
Var [Y ] =
∑ Var [Zi ] + ∑ Cov [Zi , Zj ]
i =1
i 6 =j
= v (µ)[1 + (n − 1)ρ]
A geometric treatment of overdispersion
Other
Beta-Binomial, Probit/Logistic-Binomial, ...
A geometric treatment of overdispersion
Other
Beta-Binomial, Probit/Logistic-Binomial, ...
Markov-chain Binomial (Rudolfer 1990):
A geometric treatment of overdispersion
Other
Beta-Binomial, Probit/Logistic-Binomial, ...
Markov-chain Binomial (Rudolfer 1990): Z1 , . . . , Zn are binary
rv with a general transition probability
1−α
α
β
1−β
A geometric treatment of overdispersion
Other
Beta-Binomial, Probit/Logistic-Binomial, ...
Markov-chain Binomial (Rudolfer 1990): Z1 , . . . , Zn are binary
rv with a general transition probability
1−α
α
β
1−β
The sum Y = ∑ni=1 Zi has density
g (y ; µ, φ) =
αy (1 − α)m−1−2y βy +1
S (y ; α, β, n)
α+β
E [Y ] = µ
Var [Y ] = v (µ)[1 + (n − 1)r (α, β)]
A geometric treatment of overdispersion
Overdispersed Binomial: Mixed parametrisation
Embed the extended binomial g (y ; µ, φ) into the multinomial via
g (y ; µ, φ) 7→ (f (0; µ, φ), f (1; µ, φ), . . . , f (n; µ, φ))T
A geometric treatment of overdispersion
Overdispersed Binomial: Mixed parametrisation
Embed the extended binomial g (y ; µ, φ) into the multinomial via
g (y ; µ, φ) 7→ (f (0; µ, φ), f (1; µ, φ), . . . , f (n; µ, φ))T
so density of U = (U1 , . . . , UN ) where Ui = II(Y = i ) is
exp(U> χ(µ, φ) − κ (χ))
A geometric treatment of overdispersion
Overdispersed Binomial: Mixed parametrisation
Embed the extended binomial g (y ; µ, φ) into the multinomial via
g (y ; µ, φ) 7→ (f (0; µ, φ), f (1; µ, φ), . . . , f (n; µ, φ))T
so density of U = (U1 , . . . , UN ) where Ui = II(Y = i ) is
exp(U> χ(µ, φ) − κ (χ))
where χ(µ, φ) =
(log(f (1, µ, φ)/f (0, µ, φ)), . . . , log(f (n, µ, φ)/f (0, µ, φ))). Define


N




log
(
)
1
1
T (1)




 2 
 T (2) 


 log (N2 ) 


dT = 
c
=
d= . 



..
..


 .. 


.


.
N
T (N )
N
log (N )
A geometric treatment of overdispersion
Overdispersed Binomial: Mixed parametrisation
Apply linear transformation Y = AT U where A = [d dY 2 k B ]
A geometric treatment of overdispersion
Overdispersed Binomial: Mixed parametrisation
Apply linear transformation Y = AT U where A = [d dY 2 k B ]
E [Y1 ] = E [Y ] = µ
E [Y2 ] = E [Y 2 ] = µ(2) = Var (Y ) + µ2
A geometric treatment of overdispersion
Overdispersed Binomial: Mixed parametrisation
Apply linear transformation Y = AT U where A = [d dY 2 k B ]
E [Y1 ] = E [Y ] = µ
E [Y2 ] = E [Y 2 ] = µ(2) = Var (Y ) + µ2
The density of Y is EF with natural parameter
ω = A−1 χ
A geometric treatment of overdispersion
Overdispersed Binomial: Mixed parametrisation
Apply linear transformation Y = AT U where A = [d dY 2 k B ]
E [Y1 ] = E [Y ] = µ
E [Y2 ] = E [Y 2 ] = µ(2) = Var (Y ) + µ2
The density of Y is EF with natural parameter
ω = A−1 χ
Now use mixed parametrisation
(µ, µ(2) , ω∗ (µ, φ)) or (µ, Var (Y ), ω∗ ) ω∗ ∈ IRn−2
to characterise the extended Binomial g (y ; µ, φ)
A geometric treatment of overdispersion
Overdispersed Binomial: Mixed parametrisation
Var (Y )
ω∗
Binomial
v (µ)
ω0∗
Beta-Binomial
v(µ)φ
ω0∗ + u(µ, φ)
Multiplicative
v (µ)φ
ω0∗
Multiplicative+
Var (Y )
ω0∗ + δu
Markov
v (µ)φ
ω0∗ + v(µ, φ)
Family
A geometric treatment of overdispersion
Summary
Models for overdispersed data can be expressed in terms of
simple geometrical operations
A geometric treatment of overdispersion
Summary
Models for overdispersed data can be expressed in terms of
simple geometrical operations
Goes beyond exponential and mixture family structures
A geometric treatment of overdispersion
Summary
Models for overdispersed data can be expressed in terms of
simple geometrical operations
Goes beyond exponential and mixture family structures
Work supported by EPSRC grant EP/E017878/1
A geometric treatment of overdispersion
References
Jorgensen, B. (1997) The Theory of Dispersion Models.
Chapman and Hall.
Efron, B. (1986). Double exponential families and their use in
generalized linear regression. JASA (81) 709–721
Shmueli, G. et al. (2005) A useful distribution for fitting
discrete data: revival of the ConwayMaxwellPoisson
distribution. JRSSC (54) 127–142
Altham, P.M.E (1978) Two Generalizations of the Binomial
distribution. Applied Statist. (27) 162–167
Cox, D.R. (1983) Some remarks on overdispersion.
Biometrika (70) 269–274
Rayner, J.C.W & Thas, O. & Best, D.J. (2009) Smooth
goodness of fit tests: using R. John Wiley and Sons.
Anaya-Izquierdo, K. & Marriott, P. (2007) Local mixtures of
exponential families. Bernoulli (13) 623–640
A geometric treatment of overdispersion
References
Kupper, L.L. & Haseman, J.K. (1978) The use of a correlated
Binomial model for the analysis of certain toxicological
experiments. Biometrics (34) 69–76
Rudolfer, S.M. (1990). A markov-chain model for extra
Binomial variation. Biometrika (77) 255–264
Gelfand, A.E. & Dalal, S.R. (1990) A note on overdispersed
exponential families. Biometrika (77) 55–64
Lindsay, B. (1986) Exponential family mixture models (with
Least squares estimators). Ann. Stats. (14) 124–137
Copas, J.C.B & Eguchi, S. (2005) Local model uncertainty
and incomplete-data bias. JRSSB (67) 459–513
A geometric treatment of overdispersion