Ziad Taib March 7, 2014

• “
The number of subjects in a clinical study should always be large enough to provide a reliable answer to the question(s) addressed.”
• “The sample size is usually determined by the primary objective of the trial.”
• “ Sample size calculation should be explicitly mentioned in the protocol .”
(from ICH-E9)

• Suppose we want to test if a drug is better than a placebo, or if a higher dose is better than a lower dose.
• Sample size: How many patients should we include in our clinical trial, to give ourselves a good chance of detecting any effects of the drug?
• Power: Assuming that the drug has an effect, what is the probability that our clinical trial will give a significant result?

• Sample size is contingent on design, analysis plan, and outcome
• With the wrong sample size, you will either
– Not be able to make conclusions because the study is “underpowered”
– Waste time and money because your study is larger than it needed to be to answer the question of interest

• With wrong sample size, you might have problems interpreting your result:
– Did I not find a significant result because the treatment does not work, or because my sample size is too small?
– Did the treatment REALLY work, or is the effect
I saw too small to warrant further consideration of this treatment?
– Issue of CLINICAL versus STATISTICAL significance

• Sample size ALWAYS requires the investigator to make some assumptions
– How much better do you expect the experimental therapy group to perform than the standard therapy groups?
– How much variability do we expect in measurements?
– What would be a clinically relevant improvement?
• The statistician CANNOT tell what these numbers should be
• It is the responsibility of the clinical investigators to define these parameters

E = C
E

C
A ccept H
0
OK
Type II error
Reject H
0
Type I error
OK
Ho: E=C (ineffective) H1: E≠C (effective)

Concluding for alternative hypothesis while nullhypothesis is true (false positive)
• Probability of Type I error = significance level or
 level
• Value needs to be specified in the protocol and should be small
• Explicit guidance from regulatory authorities, e.g.
0.05
• One-sided or two-sided ?
• H o : ineffective vs. H a : effective

• Concluding for null-hypothesis while alternative hypothesis is true (false negative)
• Probability of Type II error =  level
• Value to be chosen by the sponsor, and should be small
• No explicit guidance from regulatory authorities, e.g.
0.20, or 0.10
• Power
=1 -

= 1 - Type II error rate
= P(reject H
0 when H a is true)
=P(concluding that the drug is effective when it is)
• Typical values 0.80 or 0.90

• Following items should be specified
– a primary variable
– the statistical test method
– the null hypothesis; the alternative hypothesis; the study design
– the Type I error
– the Type II error
– way how to deal with treatment withdrawals

1. Continuous outcome: e.g., number of units of blood transfused, CD4 cell counts, LDL-C.
1. Binary outcome: e.g., response vs. no response, disease vs. no disease
1. Time-to-event outcome: e.g., time to progression, time to death.

• Easiest to discuss
• Sample size depends on
– Δ: difference under the null hypothesis
– α: type 1 error
– β type 2 error
– σ: standard deviation
– r: ratio of number of patients in the two groups (usually r = 1)

• Assume that we wish to study a hypothesis related to a sample of independent and identically distributed normal random variables with mean
 and standard deviation

.
Assuming
 is known, a (1-

)100% confidence interval for
 is given by
Y

Z (

2
)
 n
• The maximum error in estimating  is
E

Z (

2
)
 n which is a function of n. If we want to pre specify E, then n can be chosen according to n

Z (

2
)
2
E
2

2

• This takes care of the problem of determining the sample size required to attain a certain precision in estimating
 using a confidence interval at a certain confidence level.
Since a confidence interval is equivalent to a test, the above procedure can also be used to test a null hypothesis. Suppose thus that we wish to test the hypothesis
– H
0
: μ
1
= μ
0
– H a
: μ a
> μ
0
• with a significance level 
. For a specific alternative
H
1
: μ a
= μ
0
+
 where

>0, the power of the test is given by

• H
0
: μ
1
– μ
2
= 0
• H
1
: μ
1
– μ
2
≠ 0
• With known variances 
1 specific alternative μ
1
= μ
2 and

2
, the power against a
+
 is given by

• is normal so



P





Z (

)
Z (

2
)


 d

Z (

2
)



 d
Z
 n

 
Z (

2
)


 d

 Z (

2
)

Z (

 2
)


2 

1
2  
2
2




• In the one sided case we have n


Z (

)

Z (

 2
)

2


1
2  
2
2


For testing the difference in two means, with (equal) variance and equal allocation to each arm: n
1

2
( Z


Z

 2
)
2

2
With Unequal allocation to each arm, where n2 = n1 n
1

2 r
 r
1 ( Z


Z

 2
)
2

2
If variance unknown, replace by t-distribution

We want to test
• H
0
: P = P
0
• H a
: P >P
1 with a certain power (1-

) and with a certain significance level (

). The required sample of size is n


Z P
0
( 1

P
0
) p
2
0


Z p
1
 
 
2
P
1
( 1

P
1
)
2



We want to test
• H
0
: P
1
= P
2
• H a
: P
1
P
2 with a certain power (1-

) and with a certain significance level (

). The required sample of size is n


Z (

/ 2 ) P ( 1

P )


Z p
2
 
 p
1
P
1

2
( 1

P
1
)

P
2
( 1

P
2
)
2

,
P

( P
1

P
2
)
2

We want to to test
• H
0
:

1
=

2
= …=  k
• H a
: at least one
 i is not zero
Take:

1
 k

1 ,

2
 k ( n
Z
F
A
*
(


)
F

(


,

1
,

2
),


2

2 (

1

2

1 )


N
 k ,
 n

2


2 i k
,
2
)
2

1


(

(

1

1


2

2
)
2
2
) F
A
*

 
1

(

1
 
2
( 2

1


2


2
2
)

)( 2

2

1 ) F
A
*
 
The last equation is difficult to solve but there are tables.

StudySize

• Assume we plan a RCT aiming at comparing a new treatment (drug) with an old one (control). Such a comparison can be made in terms of the hazard ratio or some function of this. Let d stand for the drug and c for control
• In the sequel we describe the sample size required to demonstrate that the drug is better than the control treatment. This will be made under some assumptions in two steps

• Step 1: specify the number of events needed
• Step 2: specify the number of patients needed to obtain the “the right” number of events.
Step 1
In what follows we will use the notation below to give a formula specifying the number of events needed to have a certain asymptotic power in many different cases like
The log-rank test – the score test – parametric tests based on the exponential distribution – etc.
p

The proportion of individual receiveing the new drug q

1
 p

The proportion of individual receiveing the old drug d
C

 the required number of events

The critical value of the test
Z


The upper
 quantile of the standard normal distributi on

d


C


Z pq
 power
2
2

• It is not clear how  should estimated.
The difficulty lies in the fact that this depends on the exact situation at hand.
We illustrate that through some examples

• EXAMPLE 1
Assume that our observations follow exp(

) under the old drug (control) and exp(
 exp
  
) under the new drug. Then the ratio of medians of the two distributions will be
M d
M c
 ln
 e
 ln

 
 e
1

• To be able to discover a 50% increase in the median we take
M d
M c

1 e


3
2
 e
 
2
3

1. It is possible to dress a table specifying
 as a function of the remaing parameters when

=0.05 and p=q and the test is two sided. number of events.
3. The largest power is obtained when p=q.
4. When p and q are not equal divide the table value by 4pq .
5. For a one sided test, only C  needs to be changed from 1.96 to 1.64.

e
 power
0.5
0.7
0.8
0.9
1.15
1.25
787 309
1264 496
1607 631
2152 844
1.50
94
150
191
256
1.75
49
79
100
134
2.00
32
51
65
88
The number of events needed for a certain power
Under equal allocation.

• In this type of studies we are interested in testing a null hypothesis of the form
– H
0
:

T
– H a
:

T
=

P
≠

P .
against
Arm k=1
Placebo
Period j=1
T
11
Drug
Period j=2
T
21
Arm k=1
Crossover design
T
22
Arm k=2
T
12
Period j=1
Drug
Period j=2
Placebo
Arm k=2

A general model we can use for the response of the i th individual in the k th arm at the j th period is where S and e are independent random effects with means zero and standard deviations
 and

S
. The above hypotheses can be tested using the statistic T d
, which can also be used to determine the power of the test against a certain hypothesis of the type

Assuming equal sample sizes in both arms, and that
 d can be estimated from old studies we have
(*) where
 stands for the clinically meaningful difference. (*) can now be sued to determine n.

• For common situations, software is available
• Software available for purchase
– NQuery
– StudySize
– PASS
– Power and Precision
– Etc…..
• Online free software available on web search
• http://biostat.mc.vanderbilt.edu/twiki/bin/view/Main/PowerSampleSize
• http://calculators.stat.ucla.edu
• http://hedwig.mgh.harvard.edu/sample_size/size.html