The Concentration Factor
• The definition
– It measures how the students’ responses are distributed
on a multiple-choice test.
– In their response to an MC question with research-based
distracters, we assume most students will answer using
one of the various reasoning models observed in
qualitative research.
Type
I
II
III
A
20
50
100
B
20
5
0
C
20
30
0
D
20
5
0
E
20
10
0
Possible distributions of responses from a 100
students on a single 5-choice question
Random
Two models
One model
• The formulation
– The responses of a class on one question can be
represented with a vector: r = (n1, n2, …, nm),
where m = total number of choices.
m
C
m
(
m 1
2
n
 i
i 1
N
1

)
m
C has a value between 0 and 1
m
n
i 1
i
N
Model the student responses with C
Combining the C factor with the scores, we can now model the
different types of the responses:
– Low Score:Low Concentration –– LL
– Low Score:High Concentration –– LH
– etc.
Score
0~0.4
0.4~0.7
0.7~1.0
Level
L
M
H
C
0~0.2
0.2~0.5
0.5~1.0
Level
L
M
H
Three level coding scheme to model the responses
with “SC”, e.g. LH for low score and high C
• The S-C plot
S-C Boundary
1
HH
0.8
0.6
LH
MH
LM
MM
C
0.4
0.2
LL
0
0
0.2
0.4
0.6
0.8
1
S
Constraints – C gives the overall concentration including the contribution
from the score, which produces constraints on allowed regions.
The S-C State Density
%
C
%
S
N=100, m = 5
Implications of Response Types
– One-Peak: Most of the responses concentrated on
one answer. (LH or HH)
– Two-Peak: Most of the responses concentrated on
two answers, usually one correct and one incorrect.
(LM or MM)
– Non-Peak: Most of the responses somewhat evenly
distributed among three or more answers. (LL)
• Implications of the concentration factor (FCI)
Force and Motion
Answer
%
Type
5-c
58%
LM
9-c
45%
LM
18-a
63%
LM
22-c
66%
LH
28-d
51%
LM
Newton’s Third Law
Answer
%
Type
2-a
66%
LH
11-d
43%
MM
13-c
68%
LH
A LH or LM type of response often implies the
existence of a common incorrect student model related
to the physical context of the question
FCI Question #2
A
B
C
D
E
Response Type – LH
Imagine a head-on collision between a large truck and a
small compact car. During the collision:
the truck exerts a greater amount of force on the car than
the car exerts on the truck.
the car exerts a greater amount of force on the truck than
the truck exerts on the car.
neither exerts a force on the other, the car gets smashed
simply because it gets in the way of the truck.
the truck exerts a force on the car but the car doesn’t exert
a force on the truck.
the truck exerts the same amount of force on the car as the
car exerts on the truck.
FCI Question #24
Response Type – LL
A rocket drifts sideways in outer space from point "a" to point "b" as shown
below. The rocket is subject to no outside forces. Starting at position "b",
the rocket's engine is turned on and produces a constant thrust (force on the
rocket) at right angles to the line "ab". The constant thrust is maintained
until the rocket reaches a point "c" in space.
Which of the paths below best represents the path of the rocket between
points "b" and "c"?
• The Concentration of the Incorrect Responses
– The unbiased details about the distribution of the
incorrect responses can be obtained by removing the
absolute offset created by the score. (there will be no
constraints and  can be any value between 0 and 1)
m

m 1
(
m 1  1
n
i 1
2
i
 S2
(N  S)

1
m 1
)
Now the score and  are two independent variables.
Pre and post data graphical analysis
Tutorial
a)
1
Traditional
b)
1
Pre-data
Pre-data
Post-data
S-C plot for
the overall
results
Post-data
Pre-average
0.8
Pre-average
0.8
Post-average
Post-average
0.6
0.6
D
D
0.4
0.4
0.2
0.2
0
0
0
0.2
0.4
0.6
0.8
1
0
0.2
0.4
S
S- plot for
the overall
results
D
1
Overall (Traditional)
b)
1
1
0.8
0.8
0.6
D
0.6
0.4
0.4
0.2
0.2
Pre-data
0
Post-data
0
0.2
Pre-average
Post-average
0.8
S
Overall (Tutorial)
a)
0.6
0.4
0.6
S
0.8
1
Pre-data
0
Post-data
0
0.2
Pre-average
Post-average
0.4
0.6
S
0.8
1
S- plot for low
performance groups
(Tutorial)
S- plot for low
performance groups
(Traditional)
Low-Performance (Tutorial)
Low-Performance (Traditional)
1
0.8

0.6
1
2
13
0.8
22
13
2
22
18

5
0.4
28
0.4
28
18
0.6
5
9
9
0.2
Pre-data
0
Post-data
0
0.2
Pre-average
Post-average
0.2
15
15
24
24
0.4
0.6
S
0.8
1
Pre-data
0
Post-data
0
0.2
Pre-average
Post-average
0.4
0.6
S
0.8
1
S-D plot for mid
performance groups
(Tutorial)
S-D plot for mid
performance groups
(Traditional)
Mid-Performance (Traditional)
Mid-Performance (Tutorial)
1
1
0.8
0.8
0.6
0.6


0.4
0.4
0.2
0.2
Pre-data
0
Post-data
0
0.2
Pre-average
Post-average
0.4
0.6
S
0.8
1
Pre-data
0
Post-data
0
0.2
Pre-average
Post-average
0.4
0.6
S
0.8
1
Applications
• Evaluate instruments
– Compare FCI and FMCE
• Facilitate test design
–
–
–
–
Conductivity (L. Bao, M. Wittmann, and E. Redish)
Quantum (L. Bao, and E. Redish)
Astronomy (B. Hufnagel)
Wave Test (M. Wittmann)
• Facilitate instruction
– Evaluate student model condition
– Study the intermediate states in learning
(or bridging process if the instruction is such designed)