P2010 Lecture Notes
Chi-square Tests
Example problem
Suppose that the question of the handedness of psychology majors had arisen. Some persons
believed that students who choose to major in psychology are more likely to be left-handed than
persons in the general public.
Suppose that in the population of persons of college age, 12% are left handed.
The interest here is in the population of psychology majors: Is the percentage of left handed
people in that population equal to 12% or not?
Suppose a sample of 25 psychology majors was selected, and the handedness of each student was
determined. To determine handedness, a series of tasks – writing, eating with a fork, throwing a
ball overhand, etc – were described to each participant and the hand each participant reported
using for the task determined. The hand used the most (with greater weight to writing and eating
with a fork) was the “official” hand of each person.
The (hypothetical) data were as follows.
Person
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
Handedness
R
R
R
L
R
L
R
R
R
L
R
R
R
L
R
R
L
L
R
R
R
L
R
R
R
Biderman’s P201 Handouts
The percentage of left handed people in the sample on the left is
28%, far above the population proportion of 12%.
But it could be that large proportion was simply a samplespecific random deviation.
So the question is . . . How likely would a sample proportion of
28% be when the population proportion was 12%? If it would be
very Unlikely, then we could conclude that in the population of
psych majors, the percentage of left handed people is more like
28% than 12%.
On the other hand, if the probability of a sample proportion of
28% is fairly common, even when the population percentage is
12%, then we would conclude that psych majors are like the rest
of the population.
Topic 14: Chi-square Tests - 1
2/8/2016
Tests of hypotheses on population percentages:
Introduction & Relationship to Past Work
Tests for comparison of means
1) Suppose someone claims that psychology majors are more extraverted than the average
of 4 on a 1-7 scale.
Appropriate test: One Sample Z test or t-test
Is the average of the extraversion scores equal to a specified value?
2) Suppose someone claims that psychology majors are less extraverted than engineering
majors.
Appropriate test: Independent Sample t-test or Paired Sample t-test
Is the average of the extraversion scores in one population equal to the average of the
extraversion scores in a second population?
Problem
Extraversion scores are quantities – they range from 1.0 to 7.0 on the scales we use. One
person can have Extraversion = 2.3 (not very extraverted) while another can have
extraversion = 5.7 (pretty extraverted) while another can have extraversion = 3.6 (about
average) and so forth.
What if we're working with a categorical variable - one whose values represent only
qualitative, rather than quantitative distinctions between people?
Example
Opinion - Pro, Neutral, Con
Performance – Success vs. failure
Movie preference – Action, Comedy, or Slice-of-life
Political Preference – Democrat vs. Independent vs. Republican
Religious preference – Catholic, Protestant, Jewish
For such variables, it makes no sense to ask - Is the average amount of the characteristic
equal to some specific value. E.g., "Is the average amount of success equal to S.34?" or
"Is the average Religion equal to Catholic and one half?"
A new question
For these variables, our interest will probably be in the percentage of persons at each value
of the variable - e.g., “Is the percentage of left handed psych majors equal to 12% or not?”,
"Are the percentages of movie preferences for action, comedy, and slice-of-life equal or
not?” or "Are the percentages of persons in Chattanooga who are Catholic, Protestant, and
Jewish equal or not?"
Biderman’s P201 Handouts
Topic 14: Chi-square Tests - 2
2/8/2016
A new statistics: The Chi-square Statistic
For such questions - those involving numbers of persons - a test statistic different from the
t statistic or Z statistic must be employed. The statistic is called the Chi-square.
There are two types of chi-square problems: One population problems and Twopopulation problems.
One way chi-square problems first - one population
Categorical variable analog of the One-sample t-test
The first type of problem concerns the number of persons in each category of a variable in
one population. Here we're comparing percentages across the categories of a variable
within a single population. For example, consider the handedness of psychology majors –
Left vs. Right. "Is the percentage of psych majors who are left handed equal to 12% ?"
For this problem we would compute the percentage or left-handed psych majors and test
the hypothesis that the percentages were 12%
Other types of problems
Is a coin fair:
Does the Percentage of Heads and Percentage of Tails equal 50%?
Is a forest infected:
Does the % of dying trees = 2.2%, the typical % in all forests?
Car Safety:
Does the % of Cars with defects in air bags equal .00000002%, the % that
would be expected due to normal manufacturing defects?
Fairness of a die: Does % of 1s = % of 2s = % of 3s = % of 4s = % of 5s = % of 6s = 16.67%?
Each problem involves comparison of % in a category or categories of an outcome with
population percentages.
Biderman’s P201 Handouts
Topic 14: Chi-square Tests - 3
2/8/2016
One Way Chi-square Test: Overview
The situation
Each of the persons in a single population has been placed in one of the categories of a
variable. We wish to test a hypothesis about the numbers or percentages of persons in the
population in each of the categories of the variable.
General procedure
We'll take a sample from the population. Using the null hypothesis, we'll compute the
number of persons in the sample which would be expected in each category of the
variable. We'll compare these expected frequencies (symbolized with E) with the actual
observed frequencies (symbolized with O) in the categories. If the observed and expected
frequencies are "close", we'll retain the null hypothesis. But if the observed and expected
frequencies are "far" from each other, we'll reject the null.
Test Statistic: One way chi-square statistic
(O – E)2
X2

=
All
categories
The official “chi” symbol from Word: χ
(O1-E1)2
(O2-E2)2
(Ok-Ek)2
-------- = ----- + ----- + . . . ----E
E1
E2
Ek
where . . .
Summation is over all the categories of the variable.
O represents the observed frequency in a category.
E represents the expected frequency in a category.
If the null hypothesis is true, the chi-square statistic has a Chi-square ( χ2) distribution with
degrees of freedom equal to the Number of categories - 1.
That is, df = No. of categories - 1.
If the null is true, values of X2 should be close to the df value.
But if the null is false, values of X2 should be much larger than df.
Negative X2 values are impossible.
Biderman’s P201 Handouts
Topic 14: Chi-square Tests - 4
2/8/2016
One way chi-square: Worked out example problem
Problem
We’ll complete the problem used to introduce this section. The question is: Is the
percentage of left handed psychology majors equal to 12%. The data are on the first page
of this lecture handout.
Statement of Hypotheses
H0: In the population of psych majors, the percentage of those who are left handed is 12%.
H1: In the population of psych majors, the percentage of those who are left handed is not 12%.
Test Statistic: One way chi-square
Most likely value of test statistic when Null is true
If the null is true, the most likely value of X2 is df. That's because if the null is true, the Os will
be close to the expected frequencies, either slightly larger or slightly smaller than the Es.
Squaring the difference and dividing by the E yields a quantity which won't be 0, but will be
slightly positive. Summing these small positive values yields a value close to degrees of
freedom.
Values likely when null is false.
Extremely large positive values of the test statistic would be expected if the null were false.
(Negative values will never occur.)
Significance Level.
As usual, we'll set alpha = .05 (5%) This means we'll reject the null if the probability of a chisquare as large as the obtained value if the null were true is less than or equal to .05.
The results are as follows:
Conclusion
Category O
Left handed: 7
E
Right handed: 18
O-E
(O-E)2
(O-E)2/E
Left
7
12% of 25 = 3.00
4.00
16.00
16.00/ 3.00
=
Right
18
88% of 25 = 22.00
-4.00
16.00
16.00/22.00
=
5.33
0.73
----2
X = 6.06
Carry E to 2
decimal places.
Conclusion: If the null were true the probability of obtaining a chi-square as extreme as
6.06 is quite small – approximately .014. (I used my tables of p-values to get this.) So
6.06 is a value which we would NOT expect to obtain if the null were true. So we'll reject
the null. It appears that the percentage of left handed psychology majors is not 12% but
somewhere closer to 28%.
Biderman’s P201 Handouts
Topic 14: Chi-square Tests - 5
2/8/2016
Completing the Corty Hypothesis Testing Answer Sheet . . .
Give the name and the formula of the test statistic that will be employed to test the null
hypothesis.
One-Way Chi-square Test
Check the assumptions of the test
There are no distribution assumptions. Data must be categorical.
Population Proportion of Ls = .12; Proportion of Rs = .88.
Null Hypothesis:________________________________________________________________
Alternative
Population Proportions are not equal to .12 / .88.
Hypothesis:______________________________________________________________
What significance level will you use to separate "likely" value from "unlikely" values of the test
statistic?
Significance Level = _________________.05_______________________________________
What is the value of the test statistic computed from your data and the p-value?
Chi-square = 6.06
p-value < .05 from Table of p-values
What is your conclusion?
Do you reject or not reject the null hypothesis?
Reject the null. p-value is less than .050.
What are the upper and lower limits of a 95% confidence interval appropriate for the problem?
Present them in a sentence, with standard interpretive language.
Confidence intervals are not required for chi-square problems
State the implications of your conclusion for the problem you were asked to solve. That is, relate
your statistical conclusion to the problem.
Data suggest that the proportion of Left-handed persons in the population of Psych majors is
greater than .12.
Biderman’s P201 Handouts
Topic 14: Chi-square Tests - 6
2/8/2016
Example problem 2: Preferences for product color. A manufacturer of an energy drink is
trying to decide which color should be predominant on its drink labels. A number of cans of the
drink are prepared with some predominantly red, some predominantly green, and some
predominantly blue. The cans are arranged on a Walmart End-cap so that the colors are
randomly distributed. A person is hired to maintain a random distribution throughout the day.
A total of 250 cans is purchased. If color does not make a difference, then we would expect onethird of the purchased cans to be red, one-third to be green, and one-third to be blue.
On the other hand, if color was important, we would expect most of the cans to be of the
preferred color.
The number of red cans purchased was 110. The number of green, 75, and the number of blue,
65.
This is a One-way Chi-square test.
The null hypothesis is that in the population of purchasers, the preference for red, green, and blue
is 1/3. So out of 250 cans, we would expect 83.33 to be red, 83.33 to be green, and 83.33 to be
blue. (Just as the arithmetic average of a bunch of whole numbers can be a decimal, the expected
frequencies in chi-square problems can be fractional.)
Working out the problem . . .
Category O
__E__
Red
Green
Blue
83.33
83.33
83.33
110
75
65
O-E
(O-E)2
_______(O-E)2/E________
26.67
-8.33
-18.33
711.29
69.39
335.99
711.29 / 83.33 =
69.39 / 83.33 =
335.99 / 83.33 =
Chi-square = 8.54 + 0.83 + 4.03 = 13.40
From the Table of p-values with df = 2, p is less than .006
Biderman’s P201 Handouts
Topic 14: Chi-square Tests - 7
2/8/2016
8.54
0.83
4.03
Completing the Corty Hypothesis Testing Answer Sheet for color preferences problem . . .
Give the name and the formula of the test statistic that will be employed to test the null
hypothesis.
One-Way Chi-square Test
Check the assumptions of the test
There are no distribution assumptions. Data must be categorical.
Population Preferences for Red, Green, and Blue are 1/3 each.
Null Hypothesis:________________________________________________________________
Alternative
Population Preferences are NOT equal to 1/3 each..
Hypothesis:______________________________________________________________
What significance level will you use to separate "likely" value from "unlikely" values of the test
statistic?
Significance Level = _________________.05_______________________________________
What is the value of the test statistic computed from your data and the p-value?
Chi-square = 13.40
p-value < .05 from Table of p-values
What is your conclusion?
Do you reject or not reject the null hypothesis?
Reject the null. p-value is less than .050.
What are the upper and lower limits of a 95% confidence interval appropriate for the problem?
Present them in a sentence, with standard interpretive language.
Confidence intervals are not required for chi-square problems
State the implications of your conclusion for the problem you were asked to solve. That is, relate
your statistical conclusion to the problem.
Data suggest that the shoppers preferred the Red coloring for cans of this energy drink.
Biderman’s P201 Handouts
Topic 14: Chi-square Tests - 8
2/8/2016
Two way chi-square problems - two or more populations Start here on Tuesday
Categorical equivalent of the Independent Samples t-test.
A second type of question that must be answered when dealing with categorical variables
concerns a comparison of the number of persons in each category of one variable between
populations identified by categories of a second variable? For example,
"Are the percentages of persons who left handed and right handed in the population of
Psychology Majors equal to the corresponding percentages of left handed and right handed
persons in the population of Engineering Majors?"
Here, the percentage of persons in each category of the Handedness variable are being
compared between two populations identified by categories of the Major variable.
Population of Psychology Majors
Population of Engineering
Majors
Percentage of left handed Psych Majors
vs.
Percentage of left handed
Eng. Majors
Example problems
Effects of a treatment for forest infestation
Population of Trees Not Treated
Population of Trees Treated with spray
% of Trees dying
vs
% of trees dying
Effects of two drugs on patient mortality
Population who could be given Drug A
% of patients dying
Biderman’s P201 Handouts
Population who could be given Drug B
vs
% of patients dying
Topic 14: Chi-square Tests - 9
2/8/2016
Two Way Chi-square: Overview
The Situation
We have categorized persons with respect to some variable – that is we have a categorical
dependent variable. Suppose that variable is attitude toward abortion – categorized as "For",
"Neutral", or "Against."
We are interested in comparing the percentages of persons so categorized between two or more
populations. Suppose we’re comparing Males vs Females.
Now suppose we wonder whether Males attitudes and female attitudes are the same.
That is we wish to test the null hypothesis that percentages of Males "For", "Neutral", and
"Against" are equal to the percentages of Females "For", "Neutral", and "Against."
Population of Males
% of Males “For”
% of Males “Neutral”
% of Males “Against”
Population of Females
% of Females “For”
% of Females “Neutral”
% of Females “Against”
vs
vs
vs
Note the analogy to the independent groups t-test. However, here we are comparing
percentages of persons in the categories of a variable rather than comparing mean amounts of a
variable.
For such problems, it’s common to conceptualize the situation in terms of a two way table of
frequencies. The rows of the table represent the variable with respect to which the persons have
been categorized. The columns represent the populations being compared.
Populations being compared
MALES
FEMALES
For:
Category
of
Dependent
Variable
Neutral:
Against:
Biderman’s P201 Handouts
Topic 14: Chi-square Tests - 10
2/8/2016
General Procedure
1. Take a sample from each population.
2. Arrange the observations in a two way table with Observed counts in each cell of the
table.
3. Generate expected frequencies under the assumption that the null hypothesis of
equality of percentages is true. (See below for how to compute expected frequencies for this
problem.)
4. Compare Observed with Expected frequencies.
If the Observed and Expected frequencies are "close" to each other, then retain the
null hypothesis of equality of percentages across populations.
But if the Observed and Expected frequencies are "far" from each other, reject the
null hypothesis of equality of percentages across populations.
Test Statistic
The two way chi-square statistic.
(O - E)2
X2
=

--------
All cells
E
Expected frequencies are computed using the following rule:
(The cell's row total) * (The cell's column total)
Expected frequency for a cell = ----------------------------------------------------------The total number of observations in the table.
Degrees of freedom = (No. of rows in table - 1) * (No. of columns in table - 1)
If the null is true, the expected value of the chi-square statistic is the degrees of freedom value.
We would not expect values larger than df if the null were true.
Biderman’s P201 Handouts
Topic 14: Chi-square Tests - 11
2/8/2016
Two-way Chi-square: Worked out example
Suppose a sample of males and a sample of females were asked to categorize their attitudes
toward abortion.
The hypothetical data are presented in the following table.
Two Way Table of Hypothetical Frequencies
For
MALES
5
FEMALES
15
TOTAL
20
Neutral
24
8
32
Against
8
20
28
TOTAL
37
43
80
Statement of Hypotheses:
H0: The corresponding percentages in the two populations are equal.
H1: The corresponding percentages in the two populations are not equal.
Test Statistic:
Two way chi-square statistic.
Most likely value of test statistic when Null is true
If the null is true, the most likely value of X2 is df. That's because if the null is true, the observed
frequencies will be close to the expected frequencies, either slightly larger or slightly smaller
than the expected's. Squaring the difference and dividing by the expected frequency yields a
quantity which won't be 0, but will be slightly positive - about as big as the degrees of freedom
value.
df = (No. of rows in table - 1) * (No. of columns in table - 1) = (3 - 1) * (2 - 1) = 2
Values likely if the null is false.
Extremely large positive values of the test statistic would be likely if the null were false and.
Negative values will never occur.
Significance Level.
As usual, we'll set alpha = .05 (5%) This means we'll reject the null if the probability of a chisquare as large as the obtained value if the null were true is less than or equal to .05.
Biderman’s P201 Handouts
Topic 14: Chi-square Tests - 12
2/8/2016
Conclusion:
The expected frequencies
MALES
O=
5
FOR
O=
20*37
------- = 9.25
80
24
E=
-------
E=
-------
O=
8
O=
20
E=
-------
E=
-------
E=
Neutral
Against
TOTAL
FEMALES
O=
15
37
E=
O=
20*43
------- = 10.75
80
8
43
TOTAL
20
32
28
80
THE CHI-SQUARE STATISTIC IS THEN COMPUTED AS FOLLOWS
FOR
Neutral
Against
TOTAL
MALES
(5 - 9.25)2
---------9.25
(24 - 14.80)2
---------14.8
(8 - 12.95)2
---------12.95
37
FEMALES
(15 -10.75)2
---------10.75
(8 - 17.20)2
---------17.2
(20 - 15.05)2
---------15.05
43
TOTAL
20
32
28
80
DF = (NO. OF ROWS - 1) * (NO. OF COLUMNS - 1)
FOR OUR EXAMPLE, DF = (3 - 1) * (2 - 1) = 2 * 1 = 2.
THE VALUE OF THE TEST STATISTIC IS
X2 = 1.95+1.68+5.72+4.92+1.89+1.63 = 17.79
If the null were true, the probability of obtaining a chi-square as extreme as 17.79 would be
nearly 0. So we'll reject the null hypothesis. There are differences between the percentages of
males and females having pro, neutral, and con attitudes.
Biderman’s P201 Handouts
Topic 14: Chi-square Tests - 13
2/8/2016
P values for the chi-square test
Tabled entries are the probabilities of a chi-square
as large as the column header if the null were true.
chi-square
df
1
2
3
4
5
6
7
8
9
10
1.000
0.317
0.606
0.801
0.909
0.962
0.985
0.994
0.998
0.999
0.999
2.000
0.157
0.368
0.572
0.735
0.849
0.919
0.959
0.980
0.991
0.996
3.000
0.083
0.223
0.392
0.558
0.699
0.808
0.884
0.934
0.964
0.981
Biderman’s P201 Handouts
4.000
0.045
0.135
0.262
0.406
0.549
0.676
0.779
0.856
0.911
0.946
5.000
0.025
0.082
0.172
0.288
0.416
0.544
0.659
0.757
0.833
0.890
6.000
0.014
0.050
0.112
0.200
0.307
0.423
0.540
0.647
0.739
0.814
7.000
0.008
0.030
0.072
0.136
0.221
0.321
0.429
0.537
0.637
0.725
Topic 14: Chi-square Tests - 14
2/8/2016
8.000
0.004
0.018
0.046
0.092
0.157
0.239
0.333
0.434
0.534
0.628
9.000
0.002
0.011
0.029
0.061
0.110
0.174
0.253
0.343
0.438
0.532
10.00
0.001
0.006
0.018
0.041
0.076
0.125
0.189
0.266
0.351
0.441
Characterizing the difference
Express each O as the percentage within its column.
Males
For
`
Females
5 13.5%
15 34.9%
20
Neutral
24 64.9%
8 18.6%
32
Against
8 21.6%
20 46.5%
28
43
80
TOTAL
37
Female opinions are stronger than male opinions.
Biderman’s P201 Handouts
Topic 14: Chi-square Tests - 15
2/8/2016
Completing the Corty Hypothesis Testing Answer Sheet . . .
Give the name and the formula of the test statistic that will be employed to test the null
hypothesis.
Two-way Chi-square test
Check the assumptions of the test
Data are categorical.
Null Hypothesis:________________________________________________________________
Corresponding percentages are all equal across populations.
Alternative
Corresponding percentages are not all equal across populations.
Hypothesis:______________________________________________________________
What significance level will you use to separate "likely" value from "unlikely" values of the test
statistic?
Significance Level = _________________.05_______________________________________
What is the value of the test statistic computed from your data and the p-value?
X2 = 17.79
p-value L less than .006, so less than .050.
What is your conclusion?
Do you reject or not reject the null hypothesis?
Reject the null. p-value is less than .050.
What are the upper and lower limits of a 95% confidence interval appropriate for the problem?
Present them in a sentence, with standard interpretive language.
Confidence intervals are not required for chi-square problems
State the implications of your conclusion for the problem you were asked to solve. That is, relate
your statistical conclusion to the problem.
The corresponding percentages are NOT equal across sexes. Female attitudes are more
extreme.
Biderman’s P201 Handouts
Topic 14: Chi-square Tests - 16
2/8/2016
Chi-square Example problems.
1. (From Abelson, p. 48) In a study on the effects of saccharin on incidence of bladder cancer,
scientists at the Food and Drug Administration conducted research in which 23 Experimental rats
were fed a diet which consisted of 7.5% saccharin while 25 Control rats were given an
equivalent diet containing no saccharin. After two years, 7 of the 23 Experimental rats but only
1 of the Control rats had contracted bladder cancer. Test the null hypothesis of equal %’s of rats
with bladder cancer in the two populations. (Then think about the practical implications of this
study, especially in view of the fact you would have to consume 800 cans of diet drink daily for
your diet to contain 7.5% saccharin.)
Expected Frequencies
Saccharin
No Saccharin
Sach
No Sac
BC
7
1
8
3.83
4.17
No BC
16
24
40
19.17
20.83
23
25
48
2. A study investigating the effect of daily doses of aspirin on the incidence of heart attack was
conducted. About half of a large sample of (male) doctors was given a small dose of aspirin
daily over a period of months. The other half was given a placebo. The results are as follows:
Observed Frequencies
Heart Attack
No Heart Attach
Aspirin
104
10,933
11,037
Placebo
189
10,845
11,034
Marginal Totals
293
21,778
22,071
Expected Frequencies
Heart Attack
No Heart Attach
Aspirin
146.5
10,890.5
11,037
Placebo
146.5
10,887.5
11,034
293
21,778
22,071
Chi-square computations
X2 = (104-146.5)2/146.5 + (189-146.5)2/146.5 + (10,933-10,890.5)2/10,890.5 + (10,845-10,887.5)2/10,887.5
X2 =
12.33 + 12.33 + 0.17 + 0.17 = 25.00
With df = 1, p is < < .001.
Heart attack rates within the two groups.
Column proportions
Aspirin
Heart Attack
.0094
No Heart Attach
.9906
Placebo
.0171
.9829
The probability of a heart attack within the Placebo group was about twice that of the Aspirin
group.
3. A city council is considering offering benefits to same-sex partners. A council member
decides that if there is a statistically significant preference in her district “For” same-sex benefits,
she’ll vote for it. A survey of 150 persons from her district is taken. 92 are “For”; 58 are
“Against”. What should she do?
Biderman’s P201 Handouts
Topic 14: Chi-square Tests - 17
2/8/2016