The effect of differential item
functioning in anchor items on
population invariance of equating
Anne Corinne Huggins
University of Florida
Introduction
• Population invariance of equating
– Test score level
• Differential item functioning
– Item level
• Need to connect two facets of invariance
• Certain types of DIF might cause equating
dependence across forms of a test.
– Due to
(1) a differential difficulty, particularly for anchor items
(2) a second factor(s)
(3) Item parameter drift
• IRT true score equating
– A three-step process:
• Specify a true score on one form
• Find the corresponding IRT ability estimate
• Find the IRT ability estimate’s corresponding true score
on the other form
• Place the latent ability scores on each test on the
same scale
We can use the
probability formula to
get the relations.
• Scaling
– Mean-sigma
– Mean-mean
Q: How to estimate A under
Rasch model?
Here, measurement errors are ignored.
If a and b were biased, A and B will be biased.
5
– The Stocking and Lord method
• Minimize the difference in test characteristic curves
(TCCs) for anchor items
– The Haebara method
• Minimize the difference in item characteristics curves
(ICCs)
6
• Equating invariance
– Compare the whole group to multiple subgroups
– Group to overall (compare one subgroup with the
whole group)
Sigma Q is the unconditional SD of scores in
population.
7
• The factors that might influence population
invariance of IRT true score equating
– Differential anchor form DIF
– The magnitude of DIF in anchor items
– Number of DIF anchor items
– Direction of DIF in anchor items
– Ability differences between subpopulations
8
Method
• Uniform DIF
• R, BILOG-MG, SPSS
• Design
– Nonequivalent groups with anchor test (NEAT)
design
– Two forms contain 50 dichotomous items,
respectively, with 20% of common items
– Ability ~ N (0,1)
– 4500 participants (3000 in the references group)
– 3PL model
– 100 replications
9
– Manipulations on DIF in anchor items
•
•
•
•
•
Magnitude (.3, .6, .9)
% of DIF items (20%, 49%, 60%)
Directionality of DIF
Mean ability differences
Differential anchor form DIF
– Null conditions
• No DIF in anchor items and no mean ability differences
• No DIF in anchor items but had mean ability differences
10
Results—No DIF (RMSD(x))
11
No DIF, Equal mean abilities (RSD(x))
12
No DIF, Unequal mean abilities (RSD(x))
13
Why didn’t DIF show the impact here?
14
15
16
17
18
19
20
21
22
23
24
Conclusions
 Differential anchor form DIF occurs, the
magnitude of equating dependence increases.
IPD occurs for different subpopulations.
When differential anchor from DIF occurs:
Direction of DIF in anchor items shows the largest
effect on equating dependence.
The magnitude of DIF in anchor items
Number of DIF anchor items
Ability differences between subpopulations
25
Comments
• It is a good writing sample.
• Errors:
– P.9, since the study manipulated the direction of
DIF, the statement of magnitude of DIF should be
revised properly.
– The author intended to design the sample size as
the true subpopulations in the US, but the actual
sample sizes did not reflect its intends.
• When DIF or item parameter drift occurs in a specific
anchor item, this item should not be used as an
anchor.
• We usually do not talk about DIF in anchor items
because DIF detections should be done before any
equating or linking.
• When all anchor items are DIF items, the results
should be very similar with the ones without DIF
anchor items.
Future studies
• Sample size differences across score levels
• Test length, anchor length, ratio of anchor to
test length, mixed item formats
28