LECTURE 13
EPSY 625
 TEST
BIAS
 ITEM BIAS
 ITEM BIAS DETECTION
TEST BIAS


Issue arose after civil rights legislation in
the 1960s
African-americans (and other minorities)
were either denied jobs or placed in
lower category/paying jobs based on test
scores
TEST BIAS

Definition developed over time: related to
disparities in selection different from
base rates of application or of
population:
• 50% of applicants were minority
• 10% hired were minority

Prima facie evidence of discriminationmay be legal (eg. Female models for
women’s clothes)
TEST BIAS


Illegal discrimination based on series of
legal cases
Supreme court review of the issues
separated
• Hiring and economic discrimination
• Education
LEGAL CASES INVOLVING
TESTING
Griggs v. Duke Power (Supreme Court
of the U.S.,1971)

Tests used for selection of
employees may not discriminate
unfairly in form and
operation: the test must be
shown to be job-related,
specifically a relation to
successful job performance.
Albemarle Paper Company vs. Moody
(Supreme Court of the U.S. 1975)
•A
test used for selection of employees
must validated with a population
similar to that for whom it is used.
Awards for improper conduct were
given.
United States vs. State of South
Carolina (U.S. District Court,
South Carolina, 1977)
•Content
of the National Teacher
Examination was upheld as adequately
validated when disparate impact on
certification of black and white
teachers was observed. Overturned
criterion-related validity as the only
standard for validation.
Diana v. California State Board of
Education, No. C-70 37 RFP, District
Court of Northern California, 1970.

Ruled for plaintiff that IQ
testing for special education
populations must be normed on
culturally and linguistically
relevant populations and
contain no culturally biased
materials.
Golden Rule Insurance Company v.
Washburn, 419-76 Illinois Circuit
Court, 7th Ind. Cir. Ct. (1984).
Consent Decree.
•Ruling
that any item on a test that
separates protected groups is biased,
items with least difference are to be
selected. APA formally disagreed.
Larry P. v. Riles, 343 F. Supp. 1306
(N.D. Cal. 1972) (preliminary
injunction), affirmed 502 F. 2d 963 (9th
Cir. 1974), opinion issued No. C-71-2270
RFP (N.D. Cal. Ocotber 16, 1979).

Ruling for plaintiff that IQ
tests are biased.
PASE v. Hannon, 506 F. Supp. 831
(N.D. Ill. 1980)

Ruling against plaintiff
that IQ tests are not biased.
ITEM BIAS
Test
development was criticized for
generating test questions that were
systematically biased against minorities
and women
•Wording/phrasing
•Pictures/illustrations
•Scoring
Numerous examples of bias were shown
on current tests (1960s-70s)

ITEM BIAS
• Technical criteria were developed under the term
“item bias”
•Changed to Differential Item Functioning (DIF) to
account for any item that performed in an
inconsistent manner for different groups
-Did not require specific theoretical or conceptual
argument to remove item
-Subtle bias might not be easily observed
-Empirical effects differed from “apparent” bias as
determined by minority experts
ITEM BIAS DETECTION
IRT BASED – requires sophisticated
software, reserved for commercial
applications or expert researchers
CHI-SQUARE BASED- relatively
easy to use, potential with small
samples and clinical samples
ANOVA BASED- theoretically linked
but not empirically very good
IRT - BASED ITEM BIAS

Find D, where D=max[p(maj) -p(min)]
min
maj
D
CHI-SQUARE BASED

MANTEL-HAENSZEL
• LOG-ODDS: RELATIVE PROPORTIONS
CORRECT RESPONSES FOR EACH
ABILITY CATEGORY

SCHEUNEMANN
• CHI SQUARE FOR INTERACTION OF
GROUP (MIN-MAJ) BY ABILITY CATEGORY
MANTEL-HAENSZEL ITEM PROCEDURE
QUARTILES ON TOTAL TEST
1
MAJ
MIN
20%
30%
2
30%
30%
3
30%
25%
4
20%
15%
Differences in proportion are cumulated
into a chi-square statistic with 3 df
ANOVA-BASED

ANCOVA BASED:
• INTERACTION OF GROUP
(MIN-MAJ) x
COVARIATE (ABILITY)
0-1
1
Mean difficulty across
ability/achievement
scores
maj
min
0
ABILITY