Chi Square Tests
PhD Özgür Tosun
IMPORTANCE OF
EVIDENCE BASED
MEDICINE
The Study
Objective: To determine the quality of health
recommendations and claims made on popular medical talk
shows.
 Sources: Internationally syndicated medical television talk
shows that air daily (The Dr Oz Show and The Doctors).
 Interventions: Investigators randomly selected 40 episodes
of each of The Dr Oz Show and The Doctors from early
2013 and identified and evaluated all recommendations made
on each program. A group of experienced evidence reviewers
independently searched for, and evaluated as a team, evidence
to support 80 randomly selected recommendations from
each show.
 Main outcomes measures: Percentage of
recommendations that are supported by evidence as
determined by a team of experienced evidence reviewers.

Results






On average, The Dr Oz Show had 12 recommendations per
episode and The Doctors 11.
At least a case study or better evidence to support 54%
(95% confidence interval 47% to 62%) of the 160
recommendations (80 from each show).
For recommendations in The Dr Oz Show, evidence
supported 46%, contradicted 15%, and was not found for
39%.
For recommendations in The Doctors, evidence supported
63%, contradicted 14%, and was not found for 24%.
The most common recommendation category on The Dr Oz
Show was dietary advice (39%) and on The Doctors was to
consult a healthcare provider (18%).
The magnitude of benefit was described for 17% of the
recommendations on The Dr Oz Show and 11% on The
Doctors
Conclusions
Recommendations made on medical talk
shows often lack adequate information on
specific benefits or the magnitude of the
effects of these benefits.
 Approximately half of the
recommendations have either no
evidence or are contradicted by the
best available evidence.
 The public should be skeptical about
recommendations made on medical talk
shows.

A Fictional Answer for a Random
Dr. Oz’s Recommendation

Dr Oz:
◦ "Saturated fat is solid at room temperature,
so that means it's solid inside your body."

Patient:
◦ Thanks, Dr. Oz.You give the best advice.
◦ Carrots are very hard and dense, so they'll
petrify (transform into stone) your body,
turning you into an orange statue. Am I doing
it right?
Pull down that bread, kiddo!!!
CATEGORICAL
ONE SAMPLE
TWO SAMPLES
>2 SAMPLES
CATEGORICAL
ONE SAMPLE
TWO SAMPLES
Independent
Paired
>2 SAMPLES
Independent
CATEGORICAL
TWO SAMPLES
ONE SAMPLE
One sample difference
of proportions test
Independent
2 x 2 Chi
Square Test
One Sample
Chi Square
Test
>2 SAMPLES
Paired
Independent
Mc Nemar
Test
N x M Chi
Square Test
Fisher’s
Exact Test
Parametric
Nonparametric
Cross Table (Contingency Table)
• enables showing two or more variables
simultaneously in table format
• a table of counts cross-classified according to
categorical variables
• best way to include sub-group descriptive
statistics
• simplest contingency table is a 2 x 2 table
• is good for demonstrating possible
relationships among variables
Cross Table (Contingency Table)
• An r X c contingency table shows the
observed frequencies for two variables.
• The observed frequencies are arranged in r
rows and c columns.
• The intersection of a row and a column is
called a cell
Misreading the Table
• it is important to correctly read the information
given in a table
• although the original data do not change at all,
tables can be arranged in several different views
• looking at the table does not necessarily show
the reader about possible relationships among
variables
– in order to decide on the existence of relationship,
«statistical hypothesis testing» is required
Observed versus Expected
• In a cross tabulation, the actual numbers in
the cells of the table are called the observed
values
• Observed Frequencies are obtained
empirically through direct observation
• Theoretical, or Expected Frequencies are
developed on the basis of some hypothesis
Expected Frequencies
• Assuming the two variables are independent,
you can use the contingency table to find the
expected frequency for each cell.
Finding the Expected Frequency for Contingency Table Cells
The expected frequency for a cell Er,c in a contingency table is
Expected frequency E r ,c 
(Sum of row r )  (Sum of column c )
.
Sample size
Example:
Find the expected frequency for each “Male” cell in the
contingency table for the sample of 321 individuals. Assume
that the variables, age and gender, are independent.
Age
Gender 16 – 20 21 – 30 31 – 40 41 – 50 51 – 60 61 and Total
older
Male
32
51
52
43
28
10
216
Female
13
22
33
21
10
6
105
Total
45
73
85
64
38
16
321
Expected Frequency
Example continued:
Age
Gender
16 – 20
21 – 30
31 – 40
41 – 50
51 – 60
Male
Female
Total
32
13
45
51
22
73
52
33
85
43
21
64
28
10
38
Expected frequency E r ,c 
61 and
older
10
6
16
Total
216
105
321
(Sum of row r )  (Sum of column c )
Sample size
E1,1  216  45  30.28
E1,2  216  73  49.12
E1,3  216  85  57.20
E1,4  216  64  43.07
E1,5  216  38  25.57
E1,6  216  16  10.77
321
321
321
321
321
321
Chi-Square Independence Test
• A chi-square independence test is used to test the
independence of two variables.
• Using a chi-square test, you can determine whether
the occurrence of one variable affects the probability
of the occurrence of the other variable.
• For the chi-square independence test to be used, the
following must be true.
1. The observed frequencies must be obtained by using a
random sample.
2. Each expected frequency must be greater than or equal
to 5.
Chi-Square Independence Test
• We are looking for significant differences
between the observed frequencies in a table
(fo) and those that would be expected by
random chance (fe)
2 x 2 Chi Square
Second
Criteria
1
2
Total
2
2
χ  
2
i 1 j1
First criteria
+
O11
O12
Total
O1.
O21
O22
O2.
O.1
O.2
N
(Oij  E ij ) 2
E ij
E ij 
Eij should be greater than or equal to 5.
df = (r-1)(c-1)=1
O i.O.j
N
Family
History
Squint (Şaşılık)
Total
+
+
20
-
15
14.58
20.42
Total
35
30
35.42
55
49.58
85
50
70
120
Is squint more common among children with a positive family history?
Is there an association between squint and family history of squint?
χ  4.869
2
2(1,0.025)=5.024 > 4.869. Accept H0.
There is no relation between squint and family history
Attention
In 2 X 2 contingency tables,
if any expected frequencies are less than 5,
then alternative procedure to called Fisher’s
Exact Test should be performed.
An Example
• A study was conducted to analyze the relation
between coronary heart disease (CHD) and smoking.
40 patients with CHD and 50 control subjects were
randomly selected from the records and smoking
habits of these subjects were examined. Observed
values are as follows:
Observed and expected frequencies
CHD
Smoking
+
2
-
Yes
10
6.2
30 33.8
40
No
4
7.8
46 42.2
50
Total
14
76
90
2
(Oij – Eij )2
χ 2 = ∑∑
Eij
i=1 j=1
(10 – 6.2)
2
=
Total
6.2
(30 – 33.8)
2
+
33.8
(4 – 7.8)
2
+
7.8
(46 – 42.2)
2
+
42.2
= 4.95
df = (r-1)(c-1)=(2-1)(2-1)=1
2 =4. 95 > 2(1,0.05)=3.841
reject H0
Conclusion: There is a relation between CHD and
smoking.
An Example for Fisher’s Exact Test
Research question: does positive BRCA1 gene
actually affects the occurrence of breast cancer?
Since the percentage of the cells which have expected count < 5 is 50%, Fisher’s
exact test should be applied.
According to Fisher’s test, p value is 0.070
p>α
Fail to reject H0
BRCA1 gene has no affect on breast cancer
McNemar Test
• 35 patients were evaluated for arrhythmia with two
different medical devices. Is there any statistically
significant difference between the diagnose of two
devices?
Device II
Device I
Total
Arrhythmia (+)
Arrhythmia (-)
Arrhythmia (+)
10
3
13
Arrhythmia (-)
13
9
22
Total
23
12
35
The significance test for the difference between two
dependent population / McNemar test
H0: P1=P2
Ha: P1 P2
z

b – c –1
bc
3 – 13 – 1
Critical z value is ±1.96
3  13
 2.25
Reject H0
McNemar test approach:
(b – c)
 
bc
2
 
2
 
2
( b – c – 1)
bc
2(1,0.05)=3.841<5.1
2
2

( b – c – 1)
bc
( 3 – 13 – 1)
3  13
p<0.05; reject H0.
2
2
 5.1
Evaluation of arrhythmia patients using these two devices will provide significantly
different results. Further research is required to understand which one is better
for diagnosis.
N x M Chi Square
A researcher wants to know whether the mothers age is affecting the probability
of having congenital abnormality of neonatals or not. The collected data is given
in the table:
Congenital abnormality
Age groups
Present
Absent
Total
≤25
3
22
25
26-35
8
34
42
>35
18
16
34
Total
29
72
101
H0: There is no relation between the age of mother and presence of congenital
abnormality.
Under the assumption that null hypothesis is true:
(Expected count)
Congenital abnormality
Age groups
Present
Absent
≤25
3 (7.2)
22 (17.8)
26-35
8 (12.1)
34 (29.9)
>35
18 (9.8)
16 (24.2)
Reject H0
Congenital abnormality
Age
groups
χ2
Present
Absent
≤25
3 (7.2)
22 (17.8)
3.44
26-35
8 (12.1)
34 (29.9)
1,95
>35
18 (9.8)
16 (24.2)
9,64
Omit the
>35 age
group
Congenital abnormality
Age
groups
H0 is accepted
Present
Absent
≤25
3
22
26-35
8
34
At the end of the analysis, we should conclude that the risk of
having a baby with congenital abnormality is significantly higher for
>35 age group.
However, risk is not differing significantly between <= 25 age group
and 26-35 age group
Attention
In N x M contingency tables,
if the proportion of cells those have expected
frequencies less than 5 is above 20%, then it is
not possible to perform any statistical analysis
EXAMPLE: Researcher wants to know if there is any
significant difference among education groups in terms of
their alcohol consumption rates
At the end of the analysis, since the proportion of cells which have
expected count <5 is 50%, we must conclude that this hypothesis cannot
be tested under this circumstances. The samples size in the study is not
high enough.
Calculated p value is not valid.