A Mixture IRT Model Analysis
of Ethnic Group DIF
Taehoon Kang and Allan S. Cohen
University of Wisconsin-Madison
Introduction
• This study examines a method designed to help
understand what causes DIF.
• Efforts to detect DIF typically tell little about what
caused DIF.
- Most methods associate some manifest group
characteristic such as gender or ethnicity with
differential item performance.
- These characteristics are usually only weakly
related to DIF.
• We start by assuming that not all examinees in one
group are consistently advantaged (or
disadvantaged) by a DIF item.
• Instead, we assume that examinees advantaged (or
disadvantaged) by DIF items are better regarded
as latent classes in the data.
• We then use mixture IRT models for identifying
latent groups of examinees that are advantaged (or
disadvantaged) by DIF in particular items.
Data
• 1996 NAEP State Mathematics Assessment
for Grade 8, Block 4
– 21 MC items (N=18,958).
• Sample 1 : 1,000 Caucasian and 1,000
African-American examinees
• Sample 2 : 1,000 Caucasian and 1,000
Hispanic examinees
Detection of DIF
• For each sample, likelihood ratio tests were
used to detect DIF in each of the 21 items
between two manifest groups.
• Four DIF items were found in Sample 1
(C vs. AA)
• Three DIF items were found in Sample 2
(C vs. H)
Mixture 3 PL Model (M3plM)
• The mixture 3PL model (M3plM) describes the
probability of a correct response to an item as
• The 3PL is assumed to hold for each class, but the
item parameters differ for different classes, and
• each examinee is parameterized by a class
membership (g) and a within class ability
parameter (2g).
• If the latent classes correspond directly to
ethnicity, examinees in one ethnic group
would be classed together separately from
examinees in the other ethnic group:
– Sample 1: Caucasian examinees would be
classed in a “C-like” class and AfricanAmericans in an “AA-like” class.
– Sample 2: Caucasian examinees would be
classed in a “C-like” class and Hispanic
examinees would be classed in an “H-like”
class
Results: Composition of Latent Groups:
Item Characteristic Curves of DIF items
: Caucasian & African-American
Manifest groups
Latent groups
0.8
0.6
0.2
0.4
P
0.2
0.4
P
0.6
0.8
1.0
Item 5
1.0
Item 5
-2
0
2
4
-4
0
theta
theta
Item 21
Item 21
2
4
2
4
0.6
C-like
AA-like
0.0
0.2
0.4
P
0.0
0.2
0.4
P
0.6
0.8
C
AA
0.8
-2
1.0
1.0
-4
C-like
AA-like
0.0
0.0
C
AA
-4
-2
0
theta
2
4
-4
-2
0
theta
Item Characteristic Curves of DIF items
: Caucasian & Hispanic
Manifest groups
Latent groups
0.8
0.6
0.4
P
0.4
P
0.6
0.8
1.0
Item 12
1.0
Item 12
0.2
C-like
H-like
0.0
0.0
0.2
C
H
0
2
4
-4
-2
0
theta
theta
Item 14
Item 14
2
4
0.8
0.6
-4
-2
0
theta
2
4
C-like
H-like
0.0
0.0
C
H
0.2
0.4
P
0.2
0.4
P
0.6
0.8
1.0
-2
1.0
-4
-4
-2
0
theta
2
4
Results: Characteristics of Examinees
Conclusions
•
•
•
•
Ethnicity appears at best to be a weak clue to the
causes of ethnic group DIF.
M3plM gave us latent groups which are much
more homogeneous in responding to DIF items.
Results of this study suggest that at least some of
the explanation of DIF may be related to
attitudes toward mathematics.
Mixture IRT models provide further insight into
how DIF affects the responses of different latent
subpopulations of examinees.