Comparing 2 Groups
• Most Research is Interested in Comparing 2
(or more) Groups (Populations, Treatments,
Conditions)
– Longitudinal: Same subjects at different times
– Cross-sectional: Different groups of subjects
• Independent samples: No connection
between the subjects in the 2 groups
• Dependent samples: Subjects in the 2
groups are “paired” in some manner.
Explanatory Variables/Responses
• Subjects (or measurements) in a study are first
classified by which group they are in. The
variable defining the group is the explanatory or
independent variable.
• The measurement being made on the subject is
the response or dependent variable.
• Research questions are typically of the form:
Does the independent variable cause (or is
associated with) the dependent variable?
I.V.  D.V. ?????
Quantitative Responses
• For quantitative outcomes, we wish to compare 2
population means.
– Parameter: m2-m1
– Estimator:
Y 2 Y1
– Standard error:
 12  22
 Y Y 

2
1
n1
n2
– Sampling distribution : Approximately normal
2
2 



1
Y 2  Y 1 ~ N  m 2  m1 ,  Y 2 Y 1 
 2 


n
n
1
2


Large-Sample CI for m2- m1
• Independent, Large-samples: n120, n220
• Must estimate the standard error, replacing
the unknown population variances with
sample variances: ^
s12 s22
Y
2 Y 1

n1

n2
• Large-sample (1-a)100% CI for m2-m1:
Y
2

^


 Y 1  za / 2  Y 2 Y 1  Y 2  Y 1  za / 2
s12 s22

n1 n2
Significance Tests for m2- m1
• Typically we wish to test whether the two means
differ (null hypothesis being no difference, or
effect). For independent samples:
• H0: m2- m1=0 (m2= m1)
• Ha: m2- m10 (m2  m1)

Y 2  Y 1  0
Y 2 Y1
• Test Statistic:
z 

obs
^
Y
2 Y 1
s12 s22

n1 n2
• P-value: 2P(Z  |zobs|)
• For 1-sided tests (Ha: m2- m1>0) P=P(Z  zobs)
Qualitative Responses
• For quantitative outcomes, we wish to
compare 2 population proportions.
– Parameter: p2-p1
^
^
– Estimator:
p 2 p 1
– Standard error:

^
^
p 2 p 1

p 1 (1  p 1 ) p 2 (1  p 2 )
n1

n2
– Sampling distribution : Approximately normal


p
(
1

p
)
p
(
1

p
)
1
1
2
2

p 2  p 1 ~ N  p 2  p 1 ,  ^ ^ 


p 2 p 1
n
n
1
2


^
^
Large-Sample CI for p2-p1
• Independent, Large samples (see sample
size criteria from Chapter 6 for p)
• Estimated standard error of the difference in
sample proportions:
^
^
^
^
^
p
^
^
2 p 1

p 1 (1  p 1 ) p 2 (1  p 2 )

n1
• (1-a)100% CI for p2-p1:
^
^
n2
^
^
^
^
^
p 1 (1  p 1 ) p 2 (1  p 2 )
^

^

^
^

 p 2  p 1   za / 2  p 2 p 1   p 2  p 1   za / 2
n1
n2




Significance Tests for p2- p1
• Typically we wish to test whether the two
proportions differ (null hypothesis being no
difference, or effect). For independent samples:
• H0: p2- p1=0 (p2= p1)
• Ha: p2- p10 (p2  p1)
• Test Statistic:
zobs
^
^

p 2  p 1   0



^

^
^
p 2 p 1
^
^
p 2 p 1
1 1
p (1  p )  
 n1 n2 
^
^
^
^
n1 p 1  n2 p 2
p
n1  n2
^
Significance Tests for p2- p1
• P-value: 2P(Z  |zobs|)
• For 1-sided tests (Ha: p2- p1>0) P=P(Z  zobs)
• When comparing means and proportions,
confidence intervals and significance tests with
the same a levels provide the same conclusions
• Confidence intervals (when available) also
provide a range of “believable” values for the
difference in the parameter for the 2 groups
Small-Sample (1-a100% Confidence Interval
for m2m1  Normal Populations
Assuming equal population variances, “pool” sample
variances to get a better estimate of  :
(n1  1) s  (n2  1) s

n1  n2  2
2
1
^
Y
2

^
 Y 1  ta / 2, 
2
2
1 1

n1 n2
df    n1  n2  2
Small-Sample Test for m2m1
Normal Populations
• Case 1: Common Variances (12 = 22 = 2)
• Null Hypothesis:
H 0 : m1  m 2   0
• Alternative Hypotheses:
– 1-Sided:
H A : m1  m 2   0
– 2-Sided:
H A : m1  m 2   0
• Test Statistic:
tobs 
(Y 1  Y 2 )   0
^

 1
1 



 n1 n2 
^ 2

(n1  1) S12  (n2  1) S 22

n1  n2  2
Small-Sample Test for m2m1
Normal Populations
• Observed Significance Level (P-Value)
• Special Tables Needed, Printed by Statistical Software
Packages
– 1-sided alternative
• P=P(t  tobs) (From the tn distribution)
– 2-sided alternative
• P=2P( t  |tobs| ) (From the tn distribution)
• If P-Value  a, then reject the null hypothesis
Small-Sample Inference for m2m1
Normal Populations
• Case 2: 12  22
• Don’t pool variances:
^
Y
2 Y 1
s12 s22


n1 n2
• Use “adjusted” degrees of freedom (Satterthwaites’
Approximation) :
s
s 
2
1
n* 
2
2
2

n n 

2 
 1
2
2
  s2

 s22

 1

n1 
n2 

 

 n 1
n2  1
1










Fisher’s Exact Test
• Method of testing for testing whether p2=p1 when
one or both of the group sample sizes is small
• Measures (conditional on the group sizes and
number of cases with and without the
characteristic) the chances we would see
differences of this magnitude or larger in the
sample proportions, if there were no differences in
the populations
Example – Echinacea Purpurea for Colds
• Healthy adults randomized to receive EP (n1.=24)
or placebo (n2.=22, two were dropped)
• Among EP subjects, 14 of 24 developed cold after
exposure to RV-39 (58%)
• Among Placebo subjects, 18 of 22 developed cold
after exposure to RV-39 (82%)
• Out of a total of 46 subjects, 32 developed cold
• Out of a total of 46 subjects, 24 received EP
Source: Sperber, et al (2004)
Example – Echinacea Purpurea for Colds
• Conditional on 32 people
developing colds and 24
receiving EP, the
following table gives the
outcomes that would have
been as strong or stronger
evidence that EP reduced
risk of developing cold (1sided test). P-value from
SPSS is .079.
EP/Cold
Plac/Cold
14
18
13
19
12
20
11
21
10
22
Example - SPSS Output
r
C
O
L
N
o
e
o
T
E
4
P
2
T
6
a
r
c
c
p
t
t
s
a
s
d
i
i
i
l
d
d
d
f
u
b
P
0
1
4
a
C
4
1
9
L
1
1
0
F
4
9
N
6
a
C
b
0
6
Dependent (Paired) Samples
• Same individual receives each “treatment”
• Same individual observed before/after
exposure
• Individuals matched on demographic or
psychological similarities
• Often referred to as “matched pairs”
Inference Based on Paired Samples
(Crossover Designs)
• Setting: Each treatment is applied to each subject or pair
(preferably in random order)
• Data: Di is the difference in scores (Trt2-Trt1) for subject
(pair) i
• Parameter: mD - Population mean difference
• Sample Statistics:

D
n
i 1
n
Di

D  D


2
n
s
2
d
i 1
i
n 1
sD  sD2
Test Concerning mD
• Null Hypothesis: H0:mD=0
(almost always 0)
• Alternative Hypotheses:
– 1-Sided: HA: mD > 0
– 2-Sided: HA: mD  0
• Test Statistic:
t obs 
D
sD
n
Test Concerning mD
P-value: (Based on t-distribution with n=n-1 df)
1-sided alternative
P = P(t  tobs)
2-sided alternative
P = 2P(t  |tobs|)
(1-a)100% Confidence Interval for mD
 sD 
D  ta / 2,n 

 n
Example - Evaluation of Transdermal
Contraceptive Patch In Adolescents
• Subjects: Adolescent Females on O.C. who then
received Ortho Evra Patch
• Response: 5-point scores on ease of use for each
type of contraception (1=Strongly Agree)
• Data: Di = difference (O.C.-EVRA) for subject i
• Summary Statistics:
D  1.77 sD  1.48 n  13
Source: Rubinstein, et al (2004)
Example - Evaluation of Transdermal
Contraceptive Patch In Adolescents
• 2-sided test for differences in ease of use (a=0.05)
• H0:mD = 0
HA:mD  0
1.77
1.77
TS : tobs 

 4.31
1.48
0.41
13
P  2 P(t  4.31)  2(.005)  .01
95%CI : 1.77  2.179(0.41)  1.77  0.89  (0.88,2.66)
Conclude Mean Scores are higher for O.C., girls find
the Patch easier to use (low scores are better)
McNemar’s Test for Paired Samples
• Common subjects being observed under 2
conditions (2 treatments, before/after, 2 diagnostic
tests) in a crossover setting
• Two possible outcomes (Presence/Absence of
Characteristic) on each measurement
• Four possibilities for each subjects wrt outcome:
–
–
–
–
Present in both conditions
Absent in both conditions
Present in Condition 1, Absent in Condition 2
Absent in Condition 1, Present in Condition 2
McNemar’s Test for Paired Samples
Condition 1\2
Present
Absent
Present
n11
n12
Absent
n21
n22
McNemar’s Test for Paired Samples
• Data: n12 = # of pairs where the characteristic is present
in condition 1 and not 2 and n21 # where present in 2 and
not 1
• H0: Probability the outcome is Present is same for the 2
conditions (p2 = p1)
• HA: Probabilities differ for the 2 conditions (p2 = p1)
T .S . : zobs
n12  n21

n12  n21
P  val  2 P( Z | zobs |)
Example - Reporting of Silicone Breast
Implant Leakage in Revision Surgery
• Subjects - 165 women having revision surgery involving
silicone gel breast implants
• Conditions (Each being observed on all women)
– Self Report of Presence/Absence of Rupture/Leak
– Surgical Record of Presence/Absence of Rupture/Leak
L
C
G
p
o
u
t
t
S
R
9
8
7
N
5
3
8
T
4
1
5
Source: Brown and Pennello (2002), “Replacement Surgery and Silicone Gel Breast Implant Rupture”,
Journal of Women’s Health & Gender-Based Medicine, Vol. 11, pp 255-264
Example - Reporting of Silicone Breast
Implant Leakage in Revision Surgery
• H0: Tendency to report ruptures/leaks is the same
for self reports and surgical records
• HA: Tendencies differ
T .S . : zobs
n12  n21
28  5


 4.00
n12  n21
28  5
P  val  2 P( Z | zobs |)  2 P( Z  4)  2(.0000317)  0