Bayesian Hypothesis
Testing for Proportions
Antonio Nieto / Sonia Extremera / Javier Gómez
PhUSE Annual Conference, 9th-12th Oct 2011, Brighton UK
Introduction
•Tests on proportions
–Frequentist approach
If pvalue < significance level → Null hypothesis will be rejected
–Bayesian approach
Probability under any hypotheses → Comparison to see what is
the most plausible alternative
Both approaches can coexist and they
should be used in the statistical interest
Bernouilli distribution
•The variable that records the patient’s response follows
a Bernouilli distribution
–Discrete probability distribution, which takes value 1 “success”
with probability “p” and 0 “failure” with probability “1-p”
p

f ( x)  1  p  q
0

E[ x]  p
if x  1
if x  0
otherwise
Var[ x]  p  q
Bernouilli
•Considering the probability to respond is 0.60
After treatment
SUCESS
FAILURE
Binomial distribution
•Sum of “n” Bernouilli experiments
–Discrete probability distribution, which counts the sum of
successes/failures out of ‘n’ independent samples
 n  x n x
f ( x)    p  q
x  0,1...n
 x
E[ x]  n  p
Var[ x]  n  p  q
Binomial
Considering the probability to respond (p=0.60) in 10 patients then
E(x)=10 x 0.6=6 Var(x)=10 x 0.6 x 0.4=2.4
Exact confidence intervals, hypothesis tests can be calculated,
binomial could be also approximated by the Normal distribution
Frequentist approach
A possible solution: Binomial distribution will be approximated with
the Normal distribution and then taking a decision based on the
pvalue associated to the Gauss curve
H0 : p  p 0
H1 : p  p 0
where p 0 is a pre - fixed value
p0  q0 
x

pˆ   N  p0 ,

n
n 

pˆ - p 0
 Z  N (0,1)
p0  q0
n
Bayes’ theorem (1763)
• It expresses the conditional probability of a random event A given
B in terms of the conditional probability distribution of event B given
A and the marginal probability of only A
• Let {A1,A2,...,An} a set of mutually exclusive events, where the
probability of each event is different from zero. Let B any event with
known conditional probability p(B|Ai). Then, the probability of
p(Ai|B) is given by the expression:
p (B | A i )  p (A i )
p (A i | B) 
p (B)
where :
- p (Ai ) a prioriprobability
- p (B | Ai) probability of B in Ai
- p (Ai | B) posterioriprobability
Bayes’ in medicine
• Sensitivity: Probability of positive test when we know that the
person suffers the disease
• Specificity: Probability of negative test when we know that the
person does not suffer the disease
Probability of hypertension=0.2, sensitivity=91% specificity=98%
Probability to have hypertension if positive test is obtained
p=0.91 x 0.2/ (0.91 x 0.2+(1-0.98) x 0.8)=0.9192
Bayesian approach
•A priori distribution
•Sample distribution
•Posterior conjugate distribution
Beta distribution
•Continuous distribution in the interval (0,1)
(a  b) a 1
f ( x) 
x (1 - x)b1; a  0, b  0; (n)  (n  1)!
(a)  (b)
a
E[ x] 
ab
ab
Var[ x] 
(a  b  1) (a  b) 2
•Posterior Beta (a,b) where a=∑xi+α, b=n-∑xi+ ß
No ‘a priori’ information
•As initial assumption probability any value between
zero and one
Uniform (0,1)=Beta (1,1)
•Sample distribution Binomial (n,p)
•Posterior Beta (a,b) where a=∑xi+1, b=n-∑xi+1
Example 1
N=40, no prior information:
–H0: Proportion of responders is ≤40%
–H1: Proportion of responders is >60%
If 24 successes then posterior probability Beta (25,17)
H0
H1
X
p<=0.
4
p>0.
6
2
4
N
Test
4 H1 is more probable than
0 H0
Prior distribution: Uniform (0,1)
Prob. under
H0
0.005347226
Prob. under
H1
0.48303
Prior Knowledge
•Bayesian tests is enhanced when some information is
available
–Example the probability will fall [0.3-0.7]
–In values relatively high of α and ß, Beta~Normal then >95% of the
probability [m±2s]; where m=mean and s=standard deviation (s)
–By means of a moment‘s method type
•m=α / (α + ß); s2=m(1-m) / (α + ß + 1)
•α = [m2 (1-m) /s2] –m; ß = (α-mα)/m=[m (1-m)2 /s2] + m -1
•Sample distribution Binomial (n,p)
•Posterior Beta (a,b) where a=∑xi+α, b=n-∑xi+ ß
Example 2
N=40, probability will fall [0.3-0.7] with a 95% probability:
–H0: Proportion of responders is ≤40%
–H1: Proportion of responders is >60%
If 24 successes then posterior probability Beta (36,28)
H0
H1
X
p<=0.
4
p>0.
6
2
4
N
Test
4 H1 is more probable than
0 H0
Prior distribution: Beta (12,12)
Prob. under
H0
0.004406341
Prob. under
H1
0.27539
SAS® macro
Beta distribution plots
Example 1
Example 2
Example 2 (other prior)
6.5
6.5
6.0
5.0
5.0
4.5
4.5
4.0
3.5
3.0
2.5
3.5
3.0
2.5
2.0
1.5
1.5
1.0
1.0
0.5
0.5
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
0.0
0.0
1.0
Posterior (30,22)
4.0
2.0
0.0
0.0
Prior (6,6)
5.5
Posterior (26,18)
Probability density
Probability density
6.0
Prior (2,2)
5.5
0.1
0.2
0.3
0.4
X
0.7
0.8
0.9
1.0
6.5
6.0
6.0
Prior (2,6)
5.5
5.0
5.0
4.5
4.5
4.0
3.5
3.0
2.5
3.5
3.0
2.5
2.0
1.5
1.5
1.0
1.0
0.5
0.5
0.2
0.3
0.4
0.5
X
0.6
0.7
0.8
0.9
1.0
Posterior (30,18)
4.0
2.0
0.1
Prior (6,2)
5.5
Posterior (26,22)
Probability density
Probability density
0.6
X
6.5
0.0
0.0
0.5
0.0
0.0
0.1
0.2
0.3
0.4
0.5
X
0.6
0.7
0.8
0.9
1.0
Conclusion
• Bayesian tests are nowadays being increasingly
especially in the context of adaptive designs
used,
• Very important aspects are:
– Good selection of the distributions
– Clear definition of the ”a priori” information collected
• A Bayesian approach has been presented to be included in the
statistical armamentarium to test proportion hypotheses
– It can be also extended to other endpoints and distributions
Questions