Conversion to Problem Solving Laboratories
William A Schwalm, Physics and Astrophysics, University of North Dakota
Mizuho K Schwalm Math, Science and Technology, University of Minnesota-Crookston
We converted our general physics laboratories to
collaborative problem solving. This poster is the
fourth year report focusing on assessment.
Sample Lab and Assessment Problem
Sample PSL Unit with GTA Solution
Each part of each problem comprises five items to be ranked for its
importance or significance to the problem on a 7-point Likert scale,
with 1 meaning unimportant and 7 meaning essential. See the
example assessment problem (panel to the left, in lower right).
Example of assessment data analysis
Environment of Labs included in the study
Problem parts are scored against two expert templates. These
typically differ at some point, because there are alternative
solution methods. For instance one may sometimes use either
force methods or energy methods. Consider one expert template
first.
Numerical score on each item is scored in the following way.
Let x represent the choice on the scale from 1 to 7 for either
student or expert.
Distance between student and expert scores
Labs included in the project
UND
At UND about 400 (Fall) or 350 (Spring) students go through labs of
one of the two-semester sequences. These course sequences range
from minimal math to trig and algebra-based and up to the calculusbase introductory physics courses. Four to five faculty members
each semester teach these large physics classes. (approximately 80
to 130 students). There are no recitations.
=
∆
(
)
Raw score on item, relative to the current expert template, is

∆ 
s=
1 −
 × 100%
∆ max 

At UMC only one section of a small algebra-based two-semester
sequence physics course (about 20 students) is offered. The lecture
instructor also teaches lab.
Sample Assessment Problem
Lab Activities in Problem Solving Labs
The lab problems have context, like many textbook problems, and
require measurement and analysis. Groups create measurement
and analysis plans, then cooperate in taking data. At the end of the
week, each student passes in a report on each lab period, The data,
analysis and response of the in-class questions are that of the group,
but each student’s report should be written in his or her own voice.
AAPT Summer Meeting 2009, Ann Arbor, MI
Sadly, it turns out to be more difficult than one would think to get
students to recognize importance the group collaboration, even for
data-taking activities that clearly require a group effort .
We can trace back from the assessment to see what part of lab
activity needs to be changed . For instance, problems 4 and 5 have to
do with circular motion. In the following red and blue curves show pre
and post scores on 5 items of 3 parts of these two problems. Where
red rises above blue, learning went the wrong way.
Reverse learning:
For each problem part, the item scores are added for each expert
template. The template yielding the highest total score for this part
is the one chosen to compute the final item scores. This results in
a set of 15 item scores per problem (3 parts× 5 items per part )
The following histograms show pre (blue) and post (purple) for
merged data from all three first courses, fall 2008.
Students solve the measurement problems in accordance with a fivestep problem solving method: (Short Form)
1. Restate the assigned problem. Identify which of the given data
are relevant and what information is important.
2. Decide which physics concepts, perhaps which mathematical
relations, are involved. Assign letters to the relevant variables.
3. Transcribe the problem into mathematical form, dividing the
variables into (a) given data (b) target quantities, and (c)
quantities not given and not targets. These are to be eliminated
one by one. Relate these formal predictions to measurements.
4. Solve the mathematical relations, sometimes several at once, by
elimination to get the target quantities. Make the measurements.
5. Use units, limiting cases and order of magnitude estimates to
check the answers. Then translate the answers back into
statements. Analyze measurement data and compare.
It is very difficult to get students to take doing pre-lab problems
seriously. They tend to go through the motions but not to think.
Systematic Problems
In this way, a distribution of problem scores for each of five
problems is constructed for student responses to a pre test at the
start of the semester and a post test at the end.
Students bring completed pre-lab assignments to the lab. In lab they
belong to semi-permanent work groups with assigned roles and
responsibilities to the team. At the start of class there are in-class
responses to method and prediction questions. The recorder writes
group responses on a white board and reports to the class.
However conflicting demands often prevent us from visiting lab
sessions and interacting with TAs and students to observe how lab is
actually progressing. Without this reality check, TAs tend to lecture the
students on how to do the lab and thus to rescue them from the
anxiety of any real problem solving.
Use of the results for next year’s lab instruction
=
∆ max max xexpert − 1 , 7 − xexpert
UMC
Need more TAs and more TA supervision: Currently in
addition to two-hr/week TA meeting we have annual threedays summer training in late August.
xexpert − xstudent
Maximum distance
Heavy GTA work load is a major concern at UND. Labs meet twohours/week, with about 24 students and one GTA per section. Each
GTA has four sections per week. We had to streamline lab
instruction so as not to overwhelm the GTAs and still have students
get something out of the lab.
Systemic Problems
Need more TA supervision: Currently in addition to two(IV, 1,1), summer
(IV, 1, 2)
hr/week TA meeting we have annual three-days
(IV, 2, 1), (IV, 2, 3)
training in late August.
(IV, 2, 5)
And
(V, 2,with
4) Tas
However we tend to skip visit to lab sessions and
interact
and students to observe how actually lab is progressing. Without this
These
to thetend
reality check, TAs tend to lecture how to do the lab
andpoint
students
importance of central
to skip thinking.
acceleration in circular
motion, and
the do
Can’t make students seriously doing pre-lab problems.
Onlytothey
misconception that the
some action but they do not think.
vector sum of forces is
zero
circular motion.
Students tend not to recognize importance of lab
not for
to get
We can pinpoint the lab
importance of lab partner.
activities and reading
Problem VI
Problem V
assignments.
Problem I
Problem II
References
More lab related materials will be found at UND General
physics lab site. Lab website will be linked to UND Physics
home page http://www.physcis.und.edu
Assessment of Learning
Assessment does not measure
student assimilation of course
content, rather it focuses on
problem-solving process.
Assessment addresses the first
two or three steps in the problem
solving method. An example
assessment problem is given to
the right and above.
Scoring is done against expert
problem solvers, which is
consistent with the cognitive
apprenticeship paradigm. Different approaches can be taken to
solving a given problem, so
scoring is done against several
different expert templates.
Problem VI
Problem III
Acknowledgements
There are five problems each
semester. Each student gets one
at random as a pre test and one
(usually different) as a post test.
Each problem has three parts,
corresponding roughly to (a)
identifying which of the given
information is significant (b) which
concepts apply, and (c) what lab
equipment or procedures would
be appropriate.
The effect size is
determined as
Problem V
effect =
s
post
− s
pre
∆ spre
Effect sizes per problem
Prob 1 = .88,
Prob 2 = .60, Prob 3 = .65, Prob 4 = .55, Prob 5 = 1.00
We thank Minnesota Physics Education Group, particularly
Professors Kenneth Heller and Patricia Heller for their kind
encouragement and providing helpful suggestions and
reference materials for our project. We thank J.M. Pickle of St.
Cloud state for suggesting the method of assessment, and R.
Landry of UND for helping to work out the details. We also
thank J. Whitehead for making several useful suggestions. The
work has been supported by NSF CCLI award NSF DUE0510570.