Student Evaluation Differences
between Different Physics by Inquiry
Courses
Gordon J. Aubrecht, II,
Ohio State University PERG
Abstract: The setting of Physics by Inquiry (PbI) classes is the laboratory. Students do
experiments as suggested by the text as well as doing their own experiments to test predictions
they have made about nature’s behavior. This is guided inquiry, and students assessments of PbI
classes reflect important aspects of inquiry, while not supporting others. We report here on results
from different versions of Physics by Inquiry courses (properties of matter, electric circuits, and
astronomy by sight and optics) using the Laboratory Program Variables Inventory (LPVI), a Qsort instrument.
Q-sort
The Q-sort mechanism,
devised by William
Stephenson in 1935, consists
in organizing the statements
or pictures into a ranking
scheme.
Q-sort instruments can get first-hand data from
large numbers of students in a short time. Our
Q-sort instrument is called the Laboratory
Program Variables Inventory (LPVI). The
LPVI contains 25 statements characteristic of
most hands-on lab environments; the
statements describe common lab activities.
It measures what individual students think are the
most and least characteristic features of classroom
activities, not what they most like or most dislike
about the course.
Method
Students were instructed to sort the 25
statements of the modified LPVI into five
groups. Group I is considered most
descriptive of the course; Group V least
descriptive. Students are forced to rank the
statements into groups of size 2, 6, 9, 6, 2,
forming a quasi-normal distribution.
Most descriptive
to
the
Least descriptive
1. Students follow the step-by-step instructions in the laboratory
manual.
2. Questions in the laboratory manual require the interpretation
of data.
3. The instructor is concerned with the correctness of the data.
4. Students are allowed to go beyond regular laboratory
exercises and do experiments on their own.
5. Laboratory activities are used to develop concepts.
6. The instructor lectures to the whole class.
7. Students are asked to design their own experiments.
8. During laboratory students record information requested by
the instructor or the laboratory manual.
9. Laboratory sessions raise new problems or result in data that
cannot be immediately explained.
10. The instructor or laboratory manual identifies the problem to
be investigated.
11. Laboratory activities require students to solve problems.
12. The laboratory manual requires that specific questions be
answered.
13. The instructor or laboratory manual requires that students
explain why certain things happen.
14. Laboratory is used to investigate a problem that comes up in
class.
15. Laboratory experiments develop skill in the techniques or
procedures of physics.
16. Questions in the laboratory manual require that students use
evidence to back up their conclusions
17. Students discuss their data and conclusions with each other.
18. The instructor or laboratory manual asks students to state
alternative ex planations of observed phenomenon.
19. During laboratory students record the in formation they feel is
important.
20. Students propose their own explanations for observed
phenomenon.
21. Students identify problems to be investigated.
22. During laboratory students check the correctness of their work
with the instructor.
23. In di scussion with the instructor, assumptions are challenged
and conclusions must be justified.
24. Students usually know the general outcome of an experiment
before doing the experiment.
25. The instructor gives information to individuals in small groups.
Circuits PbI instructors
N = 12
average score
matrix score
17
1.09
17
0.76
1
1.03
1
0.76
5
0.83
8
0.64
8
0.64
5
0.56
23
0.62
16
0.54
21, 24
-0.32
21
-0.25
14
-0.40
14
-0.29
7
-0.47
7
-0.37
4
-0.55
4
-0.39
6
-1.07
6
-0.65
The 2 analysis
We perform a 2 analysis on M’ that
allows us to see which statements are
significantly different from randomly
chosen values. These are highlighted
below. Items are considered significant
if confidence level > 95%, or, in other
words, if P(2) < 0.05 that the result
could occur randomly.
N = 12
average score
P(2)
P(2)
matrix score
P(2)
17
1.09
0.0072
17
0.76
0.0072
1
1.03
0.039
1
0.76
0.039
5
0.83
8
0.64
0.0205
8
0.64
5
0.56
23
0.62
16
0.54
0.039
16
0.504
0.039
18
-0.18
0.0283
18
-0.22
0.0283
21, 24
-0.32
21
-0.25
14
-0.40
14
-0.29
7
-0.47
7
-0.37
4
-0.55
4
-0.39
6
-1.07
6
-0.65
0.0205
<0.0001
<0.0001
Ranking of Statements
The table shows how few of the Physics
107 instructors’ choices are significant. In
addition, some non-outlier scores are
significant: statements 16 and 18.
Distribution of matrix elements for the two greatest
outliers and the two non-outlier statements for
physics instructors.
Statement
6
17
Statement
16
18
I
II
-0.10 -0.20
0.28 0.03
I
II
-0.08 0.40
0.01 -0.24
III
-0.36
-0.09
III
-0.18
-0.18
IV
0.12
-0.24
IV
-0.15
0.40
V
0.56
0.01
V
0.01
0.01
We have taken LPVI data for all
three versions of our PbI class over
several years. They are known as
106 (properties of matter), 107
(electric circuits), and 108 (optics
and astronomy by sight).
Do all three classes’ students see
the courses (all taught the same
way) as acting identically?
Student significant LPVI statements
appearing at least in two courses (red
denotes among the top/bottom five)
Physics 106
Physics 107
1
1
4
Physics 108
4
5
5
5
6
6
6
7
7
7
13
13
13
14
14
14
16
16
17
17
17
18
18
18
21
21
21
22
22
22
23
23
23
24
24
24
Remember, only significant
statements are shown. Statements 1,
4, 5, 6, 7, 13, 14, 16, 17, 18, 21, 22,
23, and 24 are commonly seen as
very characteristic or very
uncharacteristic.
5. Laboratory activities are used to develop
concepts.
17. Students discuss their data and conclusions
with each other.
22. During laboratory students check the
correctness of their work with the instructor.
23. In discussion with the instructor, assumptions
are challenged and conclusions must be justified.
6. The instructor lectures to the whole class.
7. Students are asked to design their own
experiments.
18. The instructor or laboratory manual asks
students to state alternative explanations of
observed phenomenon.
21. Students identify problems to be investigated.
24. Students usually know the general outcome of
an experiment before doing the experiment.
Statements 1, 4, and 16 are characteristic of just two classes each.
106, 107
1. Students follow the step-by-step instructions in the laboratory
manual.
106, 108
4. Students are allowed to go beyond regular laboratory exercises
and do experiments on their own.
107, 108
16. Questions in the laboratory manual require that students use
evidence to back up their conclusions.
The LVPI is a valuable tool for use
because it provides instructors
information about how students perceive
what actually happened in a course
without the need for lengthy classroom
observations.
This work supports the usefulness of Q
methodology for assessing laboratory
courses and shows that it can be used to
provide instructors with formative
assessment of their classes.