Statistical errors, one-and two sided tests.
One-way analysis of variance.
1





General purpose . Student’s t-tests examine the mean of normal populations. To test hypotheses about the population mean, they use a teststatistic t that follows Student’s t distribution with a given degrees of freedom if the nullhypothesis is true.
One-sample t-test.
There is one sample supposed to be drawn from a normal distribtuion. We test whether the mean of a normal population is a given constant:
 H0:

=c
Paired t-test (=one-sample t-test for paired differences).
There is only one sample that has been tested twice (before and after the treatment) or when there are two samples that have been matched or "paired".We test whether the mean difference between paired observations is zero:
 H0:
 differerence
=0
Two sample t-test (or independent samples t-test).
There are two independent samples, coming from two normal populations. We test whether the two population means are equal:
 H0:

1
=

2
2




Paired t-test
(related samples) twice
1st 2nd x
1 y
1 x
2 y
2
… … x n y n


Two-sample t-test
(independent samples)
Each subject is measured once, and belongs to one group .
Group Measurement
1
1
…
1
2
2
… x
1 x
2
… x n y
1 y
2
…
2 y m
Sample size is not necessarily equal
3

Testing the mean of two independent samples from normal populations: two-sample t-test



Independent samples:
 Control group, treatment group
 Male, female
 Ill, healthy
 Young, old
 etc.
Assumptions:
H
 Independent samples : x
1
, x
2

( the
µ
2
, x
 i
-s are distributed as N( µ
1
2
).
, …, x n
,

1 and y
1
, y
) and the y
2
, …, y m i
-s are distributed as N
0
:

1
=

2,
H a
:

1

2
Evaluation of two sample t-test depends on equality of variances;
To compare the means, there are two different formulas with different degrees of freedom depending on equality of variances
4

 If H
0 is true and the variances are equal, then t

SD p x

1 n y

1 m
 x

SD p y
 n nm
 m SD
2 p

( n

1 )

SD x
2 n


( m m

2

1 )

SD y
2 has Student’s t distribution with n+m-2 degrees of freedom.


If H
0 is true and the variances are not equal, then x
 y d
 s x
2 n
 s y
2 m df

.
g
2 
( m
( n

1

1 )

( m
)

( 1
 g

1 )
2
)

( n

1 ) g
 s x
2 n has Student’s t distribution with df degrees of freedom. s x
2 n
 s y
2 m
Decision
 If |t|>t
α,df
, the difference is significant at α level, we reject H0
 If |t|<t
α,df
, the difference is not significant at α level, we do not reject H0
5






H
0
:

1
2 =

2
2
H a
:

1
2 >

2
2 (one sided test)
F: the higher variance divided by the smaller variance:
 max( min(
x
2
2 x
,
,
2 y
2 y
)
)

var var
Degrees of freedom:
 1. Sample size of the nominator-1
 2. Sample size of the denominator-1
Decision based on F-table
 If F>F level
α,table
, the two variances are significantly different at α
6

| nevező 
1
1 161.4476
2 18.51282
2 3 4 5 6 7 8 9 10
199.5 215.7073 224.5832 230.1619
233.986 236.7684 238.8827 240.5433 241.8817
19 19.16429 19.24679 19.29641 19.32953 19.35322 19.37099 19.38483
19.3959
3 10.12796 9.552094 9.276628 9.117182 9.013455 8.940645 8.886743 8.845238
8.8123 8.785525
4 7.708647 6.944272 6.591382 6.388233 6.256057 6.163132 6.094211 6.041044 5.998779 5.964371
5 6.607891 5.786135 5.409451 5.192168 5.050329 4.950288 4.875872
4.81832 4.772466 4.735063
6 5.987378 5.143253 4.757063 4.533677 4.387374 4.283866 4.206658 4.146804 4.099016 4.059963
7 5.591448 4.737414 4.346831 4.120312 3.971523 3.865969 3.787044 3.725725 3.676675 3.636523
8 5.317655
4.45897 4.066181 3.837853 3.687499
3.58058 3.500464 3.438101
3.38813 3.347163
9 5.117355 4.256495 3.862548 3.633089 3.481659 3.373754 3.292746 3.229583 3.178893
3.13728
10 4.964603 4.102821 3.708265
3.47805 3.325835 3.217175 3.135465 3.071658 3.020383 2.978237
7

Control group
170
160
150
150
180
170
160
160 n=8 x =162.5 s x
=10.351 s x
2 =107.14 t

Treated group
120
130
120
130
110
130
140
150
130
120 n=10 y =128 s y
=11.35 s y
2 =128.88 s p
2 
.
 
.
F


,
Degrees of freedom 10-1=9, 8-1=7, critical value int he F-table is F 
,9,7
=3.68.
As 1.2029<3.68, the two variances are considered to be equal, the difference is not significanr.

.
16

.
  
.

.
18
Our computed test statistic t = 6.6569 , the critical value int he table t
0.025,16
=2.12. As 6.6569>2.12, we reject the null hypothesis and we say that the difference of the two treatment means is significant at 5% level
8

BP
cs oport
Kontroll
Kezelt
Group Statistics
N Mean
8 162.5000
10 128.0000
Std. Deviation
10.35098
11.35292
Std. Error
Mean
3.65963
3.59011
BP Equal variances as sumed
Equal variances not ass umed
Levene's Test for
Equality of Variances
F
.008
Sig.
.930
t
Independent Samples Test
6.657
6.730
df
16 t-test for Equality of Means
Sig. (2-tailed)
.000
Mean
Difference
34.50000
Std. Error
Difference
95% Confidence
Interval of the
Difference
Lower Upper
5.18260
23.51337
45.48663
15.669
.000
34.50000
5.12657
23.61347
45.38653
9






A study was conducted to determine weight loss, body composition, etc. in obese women before and after 12 weeks in two groups:
Group I. treatment with a very-low-calorie diet .
Group II. no diet
Volunteers were randomly assigned to one of these groups.
We wish to know if these data provide sufficient evidence to allow us to conclude that the treatment is effective in causing weight reduction in obese women compared to no treatment.
10

Group
Diet
Patient
1
2
7
8
9
10
3
4
5
6
Mean
SD
No diet
Mean
SD
17
18
19
20
21
11
12
13
14
15
16
Change in body weight
6
4
0
1
-1
5
3
10
6
6
4.
3.333
5
0
-2
-2
3
2
0
1
0
3
1
1
2.145
11



HO:  diet
=
 control,
(the mean change in body weights are the same in populations)
H a
:
 diet
 control
(the mean change in body weights are different in the populations)
 Assumptions:
 normality (now it cannot be checked because of small sample size)
 Equality of variances (check: visually compare the two standard deviations)
12

 Assuming equal variances, compute the t test- statistic: t=2.477
t
 s p x
 y
1 n

1 m
 x
 y
 s p n nm
 m

4

1
9

3 .
3333 2

10

2 .
145 2
9

10
10

11
10

11

3
9 9 .
999

4 6 .
01025
19
5 .
238

2 .
477




Degrees of freedom: 10+11-2=19
Critical t-value: t
0.05,19
=2.093
Comparison and decision:
 |t|=2.477>2.093(= t
0.05,19
), the difference is significant at 5% level p=0.023<0.05 the difference is significant at 5% level
13

Group Statistics
Change in body mass group
Diet
Control
N
10
11
Mean
4.0000
1.0000
Std. Deviation
3.33333
2.14476
Std. Error
Mean
1.05409
.64667
Change in body mas s Equal variances ass umed
Equal variances not as sumed
Independent Samples Test
Levene's Test for
Equality of Variances
F
1.888
Sig.
.185
t
2.477
2.426
t-test for Equality of Means df
19
15.122
Sig. (2-tailed)
.023
.028
Mean
Difference
3.00000
3.00000
Std. Error
Difference
1.21119
1.23665
95% Confidence
Interval of the
Difference
Lower
.46495
Upper
5.53505
.36600
5.63400
Comparison of variances.
p=0.185>0.05, not significant.
We accept the equality of variances
Comparison of means (t-test).
1st row: equal variances assumed. t=2.477, df=19, p=0.023
The difference in mean weight loss is significant at 5% level
Comparison of means (t-test). 2nd row: equal variances not assumed.
As the equality of variances was accepted, we do not use the results from this row.
14





Two lecturers argue about the mean age of the first year medical students. Is the mean age for boys and girls the same or not?
 Lecturer#1 claims that the mean age boys and girls is the same.
 Lecturer#2 does not agree.
 Who is right?
Statistically speaking: there are two populations:
 the set of ALL first year boy medical students (anywhere, any time)
 the set of ALL first year girl medical students (anywhere, any time)
Lecturer#1 claims that the population means are equal:
μ boys
= μ girls
.
Lecturer#2 claims that the population means are not equal:
μ boyys
≠ μ girls
.
15

Answer to the motivated example (mean age of boys and girls)
Age in years
Sex
Male
Female
N
Group Statistics
84
53
Mean
21.18
20.38
Std. Deviation
3.025
3.108
Std. Error
Mean
.330
.427
 The mean age of boys is a litlle bit higher than the mean age of girls.
The standard deviations are similar.
Independent Samples Test
Levene's Test for
Equality of Variances t-test for Equality of Means
Age in years Equal variances ass umed
Equal variances not as sumed
F
.109
Sig.
.741
t
1.505
1.496
df
135
108.444
Sig. (2-tailed)
.135
.138
Mean
Difference
.807
.807
Std. Error
Difference
.536
.540
95% Confidence
Interval of the
Difference
Lower Upper
-.253
1.868
-.262
1.877


Comparison of variances (F test for the equality of variances): p=0.741>0.05, not significant, we accept the equality of variances.
Comparison of means: according to the formula for equal variances, t=1.505. df=135, p=0.135. So p>0.05, the difference is not significant.
Althogh the experiencedd difference between the mean age of boys and girls is 0.816 years, this is statistically not significant at 5% level. We cannot show that the mean age of boay and girls are different.
16

17




Two tailed test
H
0
: there is no change

1
=

2,
H a
: There is change (in either direction)

1

2



One-tailed test
H
0
: the change is negative or zero

1
≤ 
2
H a
: the change is positive (in one direction)

1
>

2
Critical values are different.
p-values: p( one-tailed)= p( two-tailed)/2
18




Significant difference – if we claim that there is a difference (effect), the probability of mistake is small
(maximum

- Type I error ).
Not significant difference – we say that there is not enough information to show difference. Perhaps
 there is no difference
 There is a difference but the sample size is small
 The dispersion is big
 The method was wrong
Even is case of a statistically significant difference one has to think about its biological meaning
19

Truth
H
0 is true
H a is true do not reject H
0 correct
Decision reject H
0
(significance)
Type I. error its probability:

Type II. error its probability:
 correct
20



The probability of type I error is known (

).
The probability of type II error is not known (

)


It depends on
 The significance level (

),
 Sample size,
 The standard deviation(s)
 The true difference between populations
 others (type of the test, assumptions, design, ..)
The power of a test: 1-

It is the ability to detect a real effect
21


22

23

Mean and SD of samples drawn from a normal population N(120, 10 2 ), (i.e.

=120 and σ=10)
140
120
100
80
60
40
20
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 ismétlés

s4 s5 s6 s7
Variable s1 s2 s3 s8 s9
T-test for Dependent Samples: p-levels (veletlen)
Marked differences are significant at p < .05000
s10 s11 s12 s13 s14 s15 s16 s17 s18 s19 s20
0.304079 0.074848 0.781733 0.158725 0.222719 0.151234 0.211068
0.028262
0.656754
0.048789
0.223011
0.943854 0.326930 0.445107 0.450032 0.799243 0.468494 0.732896 0.351088 0.589838 0.312418 0.842927
0.364699 0.100137 0.834580 0.151618 0.300773 0.152977 0.201040 0.136636 0.712107 0.092788 0.348997
0.335090 0.912599 0.069544 0.811846 0.490904 0.646731 0.521377 0.994535 0.172866 0.977253 0.338436
140 0.492617 0.139655 0.998307 0.236234 0.420637 0.186481 0.362948 0.143886 0.865791 0.147245 0.399857
0.904803 0.285200 0.592160 0.429882 0.774524 0.494163 0.674732 0.392792 0.707867 0.330132 0.796021
120
0.157564 0.877797 0.053752 0.631788 0.361012 0.525993 0.352391 0.796860 0.092615 0.818709 0.263511
0.462223 0.858911 0.156711 0.878890 0.624123 0.789486 0.569877 0.932053 0.136004 0.923581 0.564532
100
0.419912
0.040189
0.875361 0.167441 0.357668 0.173977 0.258794 0.099488 0.757767 0.068799 0.371769
80
p-values (detail)
60
40
20
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 ismétlés
17 18 19 20

Knotted ropes: each knot is safe with 95% probability



The probability that two knots are „safe”
=0.95*0.95 =0.9025~90%
The probability that 20 knots are „safe”
=0.95
20 =0.358~36%
The probability of a crash in case of 20 knots ~64%


 It can be shown that when t tests are used to test for differences between multiple groups, the chance of mistakenly declaring significance
(Type I Error) is increasing. For example, in the case of 5 groups, if no overall differences exist between any of the groups, using two-sample t tests pair wise, we would have about 30% chance of declaring at least one difference significant, instead of 5% chance.
In general, the t test can be used to test the hypothesis that two group means are not different. To test the hypothesis that three ore more group means are not different, analysis of variance should be used.
27

 False positive rate for each test = 0.05


Probability of incorrectly rejecting ≥ 1 hypothesis out of N testings
= 1 – (1-0.05) N
28



In a study (Farkas channel –blocking et al
Motivating example
, 2003.) the effects of three Na+ drugs — quinidine, lidocaine and flecainide — was examined on length of QT interval and on the heart rate before and during regional ischemia in isolated rat hearts.
The table and the figure show the length of the QT intervals measured in the 4 groups. Is there a significant difference between the means?
100
90
80
70
60
50 mean
SD
Control Quinidine Lidocaine Flecainide
61
53
76
84
65
56
69
65
68
66
54
89
78
81
76
72
66
73
71
61
60.4
6.80
89
82.8
5.49
69
67.3
6.86
69
68.0
4.34
40
Kontroll Quinidine Lidocaine Flecainide

Comparison of the mean of several normal populations


Let us suppose that we have t independent samples ( t
“treatment” groups) drawn from normal populations with equal variances ~N(
µ i
,

) .
Assumptions:
 Independent samples
 normality
 Equal variances
 Null hypothesis: population means are equal,
µ
1
= µ
2
=.. = µ t
30






If the null hypothesis is true, then the populations are the same: they are normal, and they have the same mean and the same variance. This common variance is estimated in two distinct ways:
 between-groups variance
 within-groups variance
If the null hypothesis is true, then these two distinct estimates of the variance should be equal
‘New’ (and equivalent) null hypothesis: 
2 between
=

2 within their equality can be tested by an F ratio test
The p-value of this test:
 if p>0.05, then we accept H
0
 if p<0.05, then we reject H
0
. The analysis is complete.
at 0.05 level. There is at least one group-mean different from one of the others
31

Source of variation
Between groups
Within groups
Total
Sum of squares
Q k
 i t 

1 n i
( x i
 x )
2
Degrees of freedom t -1
Q b
 i t 

1 j n i 

1
( x ij
 x i
) 2
Q
 i t 

1 j n i 

1
( x ij
 x )
2
ANOVA
QT
Between Groups
Within Groups
Total
Sum of
Squares
1515.590
665.367
2180.957
N-t
N -1 df
3
19
22
Mean Square
505.197
35.019
F
14.426
Variance s k
2  t
Q k

1 s b
2 
N
Q b
 t
Sig.
.000
F p
F
 s k
2 s b
2 p
F(3,19)=14.426, p<0.001, the difference is significant at 5% level,
There are one or more different group-means
32



33



As the two-sample t-test is inappropriate to do this, there are special tests for multiple comparisons that keep the probability of Type I error as

. The most often used multiple comparisons are the modified t-tests.
Modified t-tests (LSD)

Bonferroni:
α/(number of comparisons)
 Scheffé
 Tukey
 Dunnett: a test comparing a given group (control) with the others
 ….
Control – Quinidine
Control – Lidocaine
Control – Flecainide
Mean difference
22.4333
6.9333
7.6000
Resutl of the Dunnett test p
.000
.158
.113
34












 decr
The null- and alternative hypothesis of the two-sample t-test
The assumption of the two-sample t-test
Comparison of variances
F-test
Testing significance based on t-statistic
Testing significance based on p-value
Meaning of the p-value
One-and two tailed tests
Type I error and its probability
Type II error and its probability
The power of a test
In a study, the effect of Calcium was examined to the blood pressure. The decrease of the blood pressure was compared in two groups. Interpret the SPSS results
Group Statistics treat
Calcium
Placebo
N
10
11
Mean
5.0000
-.2727
Std. Deviation
8.74325
5.90069
Std. Error
Mean
2.76486
1.77913
Independent Samples Test
Levene's Test for
Equality of Variances t-test for Equality of Means decr Equal variances ass umed
Equal variances not as sumed
F
4.351
Sig.
.051
t
1.634
1.604
df
19
15.591
Sig. (2-tailed)
.119
.129
Mean
Difference
5.27273
5.27273
Std. Error
Difference
3.22667
3.28782
95% Confidence
Interval of the
Difference
Lower Upper
-1.48077
12.02622
-1.71204
12.25749
35









One-and two tailed tests
The type I error and its probability
The type II error and its probability
The increase of Type I. error
The aim and the nullhypothesis of one-way
ANOVA
The assumptions of one-way ANOVA
The ANOVA table
Pair-wise comparisons
36