Item Analysis
What makes a question good???
Answer options?
Multiple Choice Questions
Elements of a good multiple choice
question????
Item Analysis
How can we determine if it is a good
question?????
 Distractor power
 Item difficulty
 Item discrimination
Distractor power
Strength of each distractor (should be equal)
# answering incorrect
# of distractors
E.G.
There are 4 possible answers
 Option a is correct
 if 9 people choose option b (9/3 = 3)
 and 12 people choose option c (12/3 = 4)
 and 15 people choose option d (15/3 = 5)
 then option d is the strongest distractor

Item Difficulty = p
(percent of people passing)
p = number answering correctly
total number answering
 If 100 people take the test and 34 get
question 1 correct, what is the difficulty of
that question?
p1 = 34/100 = .34
 For 65 correct p would be 65/100 = .65
SO… the higher the p, the easier the question
(score must be between 0 and 1)
Item Difficulty is...
A behavioral measure
 Based on the current group
 Allows internal comparison
 Also allows external comparison

In construct based tests
Item difficulty is similar to item
endorsement
 Number agreeing with an item

Item endorsement also...
A behavioral measure
 Based on the current group
 Allows internal comparison
 Also allows external comparison

Item discrimination = d
Does the question discriminate between those
who do well overall and those who do
poorly?
(or on a psychological test, those scoring high
or low on the construct)
Distribution
To determine level
upper (U) = top 25 to 33 % of scores
lower (L) = bottom 25 to 33 % of scores
Determining item discrimination
d = U-L
n
where d = item discrimination
U = number in the upper group who get
question correct
L = number in the lower group who get
question correct
n = number in one group (U or L)
Examples of d
With 33 in U and L
 if all in U and all in L get it correct
33 - 33 / 33 = 0
 if all in U get it correct, and none in L
33 - 0 / 33 = 1
 if none in U get it correct, and all in L
0 - 33 / 33 = -1
 if 30 in U and 10 in L get it correct
30 - 10 / 33 = .61

Value of d
Range for d = -1 to 1
 higher d indicates more discriminating (1 =
all in upper group got it correct and none in
lower group)
 negative d indicates a really bad question
(-1 indicates all in lower group got it correct
and none in upper group)
