Assessing learned problem solving behavior in the cognitive apprenticeship paradigm
William A Schwalm and Mizuho K Schwalm
Physics and Astrophysics, University of North Dakota-Grand Forks
General physics laboratories at UND have been converted to
collaborative problem solving. This poster reports on five
years after the completion of NSF CCLI conversion project.
Environment of Labs included in the study
Labs included in the project
At UND about 400 (Fall) or 350 (Spring) students go through labs of one of the twosemester sequences. These sequences range from minimal math up to the calculusbase introductory courses. Four to five faculty members each semester teach these
large physics classes. (approximately 80 to 130 students). There are no recitations.
Heavy GTA work load is a major concern at UND. Labs meet two-hours/week, with
maximum 21 students and one GTA per section. Each GTA has four sections per
week. We have had to streamline lab instruction so as not to overwhelm the GTAs
and still have students get something out of the lab.
Lab Activities in Problem Solving Labs and Use of Whiteboard
Students bring completed pre-lab assignments to the lab. These pre-labs are
checked on the way in. In lab they belong to semi-permanent work groups with
assigned roles and responsibilities to the team. At the start of class there are in-class
responses to method and prediction questions. For each group the student assigned
as recorder writes group responses on a white board and reports to the class.
Students solve the measurement problems following a five-step method:
Five-Step Problem Solving Method
1. Restate the assigned problem. Identify which of the given data are relevant and
what information is important.
2. Decide which physics concepts, perhaps which mathematical relations, are
involved. Assign letters to the relevant variables.
3. Transcribe the problem into mathematical form, dividing the variables into (a)
given data or measurable quantities. (b) target quantities, and (c) quantities not
given and not targets. The latter are to be eliminated one by one.
4. Solve the mathematical relations, sometimes several at once, by elimination to get
the target quantities. Make the measurements.
5. Use units, limiting cases and order of magnitude estimates to check the answers.
Then translate back into statements. Analyze measurement data and compare.
Problems have context, like many textbook problems, and require measurement and
analysis. Groups create measurement and analysis plans, then cooperate in taking
data. At the end of the week, each student passes in a report on each lab period,
The data, analysis and response of the in-class questions are that of the group, but
each student’s report should be written in his or her own voice.
Importance of dedicated TA training
Five years ago, we realized the need more TAs and more TA
supervision: Currently in addition to two-hr./week TA meeting we
have annual three-day summer training workshop in late August.
•Conflicting demands often prevent us from visiting lab sessions and
interacting with TAs and students to observe how lab is actually progressing.
Without this reality check, TAs tend to tell the students how to do the lab and
thus to rescue them from the anxiety of any real problem solving. When this
happens, labs fail to make students confront choices.
•It is difficult to get students to take pre-lab problems seriously. They tend to
go through the motions but not to think. TAs must work to correct this.
•Sadly, it turns out to be more difficult, sometimes, than one would think to
get students to recognize importance the group collaboration, even for datataking activities that clearly require a group effort .
At least students learn to accept assigned groups (not chosen by themselves)
This is a form of actual progress in rural states.
Summer annual 3-day long GTA training
During the summer 2009 we held the first formal summer TA training. The
focus is shifting more on hands-on participation in two =-hour Lab periods
On Day One after an introduction to CPSL GTAs work through three of first
semester labs lead by a faculty member or lead TAs. The mode is about the
same as regular lab session, with the faculty modeling the TA’s role. Use of
whiteboard is one of the things we emphasize. Each group member is asked
to wear a role tag (Manager, skeptic, recorder) to emphasize the importance
of assigned roles. Video capture/analysis has been used as an essential tool
for the study of motion. This is introduced to the TAs during the second day.
The rest of the time is dedicated for the second semester content,
particularly electricity and magnetism. The TAs are also assigned readings
on various aspects of pedagogy and the readings are discussed and/or
demonstrated in the workshop. Again GTAs practice the CPSL process. The
final hour of the training session is used for assessment.
Weekly TA meeting
At UND there are three different level of two semester-sequence
introductory physics labs are offered. In all each week we cover four
different lab exercises under this program. Though each GTAs are doing
different labs usually we choose about three different lab exercise to go over
key points. Obviously first year GTA must understand the lab exercise.
However, the goal of weekly TA meetings is to learn how to deliver. Again
senior GTA and faculty member lead the session and make sure everyone is
ready for the lab for the following week.
The test instrument consists of 10 problems, three
parts each, 5 items per part. (Available, on
request)
Normalized gains in three courses
Assessing Problem Solving Ability
•Cognitive apprenticeship paradigm
•Minnesota 5-step problems solving method:
1. Focus the problem
2. Describe the physics
3. Plan a solution
4. Execute a plan
5. Evaluate the solution
To assess the first two steps
1. Focus the problem, identify what’s called for and which
data given or implied will be important for understanding
and solving the problem.
2. Describe the physics, Identify the relevant resources,
physics ideas or physical measuring equipment that will
be needed.
Assessment does not measure student assimilation of
course content, rather is focused on the problem-solving
process.
Scoring is done against expert problem solvers, which is
consistent with the cognitive apprenticeship paradigm.
Different approaches can be taken to solving a given
problem, so scoring is done against several different
expert templates.
There are five problems for each of two semesters. Each
student gets one at random as a pre test and one (usually
different) as a post test.
Each problem has three parts, corresponding roughly to
(a) identifying which of the given information is
significant
(b) which concepts apply, and
(c) what lab equipment or procedures would be
appropriate.
More lab related materials are available at UND Physics and Astrophysics
web site under LINKs.
AAPT Summer 2012 Meeting, Philadelphia, PA
post - pre
Normalized Gain =
max - pre
Over-all Averaged Gains
15.4% 17.4% 21.4%
Acknowledgements
We thank Minnesota Physics Education Group, particularly Kenneth Heller
and Patricia Heller for suggestions and reference materials for the project.
We thank J.M. Pickle of St. Cloud state for suggesting the assessment, and
R. Landry for helping with details. Supported by NSF CCLI award NSF
DUE-0510570.