Test Item Influences and
Dependencies
G. Donald Allen
Department of Mathematics
Texas A&M University
Test Item Influences and
Dependencies
1
Given and n item test
• We can arrange the items in any order we
wish.
• Problem. What is the best strategy for
arranging problems on a test?
– Placing the easy items first followed by the more
difficult ones?
– Randomly placing items?
– Cluster difficult and easy items separately?
Test Item Influences and
Dependencies
2
The Situation
• Additional assumptions are needed, the
foremost being some model of the effect of
one question upon succeeding ones.
• For example, if the test begins with a very
difficult item, one which most students will not
solve, will this event affect their state of mind,
or confidence, toward solving subsequent
problems?
Test Item Influences and
Dependencies
3
Goal
• What we intend to show is that if we do allow
test item dependencies, there can results a
substantial variation on test scores depending
on the ordering of the test item.
Test Item Influences and
Dependencies
4
Toward a model
Test Item Influences and
Dependencies
5
This means…
Test Item Influences and
Dependencies
6
The grade
Test Item Influences and
Dependencies
7
Influence and Dependencies
Hypotheses
Test Item Influences and
Dependencies
8
Parameters of Influence
Test Item Influences and
Dependencies
9
The Threshold
Test Item Influences and
Dependencies
10
Positive or Negative?
Test Item Influences and
Dependencies
11
The factors
Test Item Influences and
Dependencies
12
Boost & Fade
Conditinoal
Probability
1
1 
1  1
p2
p
2

1  1
p2

1
1
c
Confidence
Test Item Influences and
Dependencies
13
Conditional Probabilities
p2
PP 2 | P 1
if c i  i
1 1 1 p 2 if c i 1
01 1 1 p 2 if c i 0
Test Item Influences and
Dependencies
14
Psychological factors
• In fact α and β are psychological factors of
emotional responses to test questions.
• For poor test takers α may be larger than β, and
vice-versa for good test takers.
• Using different α and β here makes the
problem very difficult.
Assumption 1
a i i , i 1, . . . , n
Test Item Influences and
Dependencies
15
Rewriting the model
Test Item Influences and
Dependencies
16
Conditional probability
Test Item Influences and
Dependencies
17
The Two Question Test
Test Item Influences and
Dependencies
18
Two Cases
Test Item Influences and
Dependencies
19
What to compare?
Quantities are not comparable
as is, without more information
Test Item Influences and
Dependencies
20
More Assumptions
Test Item Influences and
Dependencies
21
Easy precedes Difficult
Test Item Influences and
Dependencies
22
Many Question Test
•
•
•
•
Multiple Questions
Multiple difficulties
Constant β’s
Constant γ’s
Test Item Influences and
Dependencies
23
Many Question Test
• Two possible models
– Simplistic. Question i influences Question i+1
only
– More realistic. The influence for question i+1
comes from all preceding questions, though
perhaps with fading memory.
Test Item Influences and
Dependencies
24
Simple Model Grade
Test Item Influences and
Dependencies
25
Optimize
Test Item Influences and
Dependencies
26
Strategies and Observations
Test Item Influences and
Dependencies
27
Expanded version
Test Item Influences and
Dependencies
28
An Interesting Connection
Traveling Salesman problem
Test Item Influences and
Dependencies
29
Traveling Salesman
Test Item Influences and
Dependencies
30
The Two Difficulties Test
Test Item Influences and
Dependencies
31
Main Result
Test Item Influences and
Dependencies
32
Proof (sketch)
Test Item Influences and
Dependencies
33
Test Item Influences and
Dependencies
34
Test Item Influences and
Dependencies
35
Step two.
Test Item Influences and
Dependencies
36
Test Item Influences and
Dependencies
37
Step Three
Test Item Influences and
Dependencies
38
Test Item Influences and
Dependencies
39
Avoiding every single detail, we
have…
Test Item Influences and
Dependencies
40
Multiple Item Difficulties
• Here there are no clear answers.
• It seems somewhat dependent on gamma,
unless
• Essentially, we can demonstrate there is no
universal algorithm for a priori arranging the
test items.
Test Item Influences and
Dependencies
41
Ordering the Other Test Items
Test Item Influences and
Dependencies
42
Local Strategy
Test Item Influences and
Dependencies
43
A Weaker Local Result
Test Item Influences and
Dependencies
44
Case 1
Test Item Influences and
Dependencies
45
Case 2
Test Item Influences and
Dependencies
46
Case 3
Test Item Influences and
Dependencies
47
Test Item Influences and
Dependencies
48
Test Item Influences and
Dependencies
49
Test Item Influences and
Dependencies
50