Teaching and Learning Physical
Science in Urban Secondary Schools:
Assessing the Assessments
Joan Whipp, Michael Politano, Michele Korb – Marquette University
Mel Sabella, Chicago State University
David Hammer, University of Maryland
Kristi Gettelman, Wisconsin Conservatory of Lifelong Learning
Kelly Kushner, Shorewood Intermediate School
Megan Ferger, Roosevelt Middle School
Modeling in Physical Science for
Urban Teachers Grades 6-10
• 2004-2006
• 18 middle and high school science teachers
from ten urban Milwaukee schools
• 2 summer institutes; 8 follow-up half-day
modeling workshops; an online community;
classroom mentoring
• Physical Science Modeling Workshop
curriculum developed at Arizona State
• Focus on increasing teacher content
knowledge and instructional strategies
Assessing Content
Knowledge
Physical Science Concept Inventory
(PSCI)
Multiple-choice diagnostic test based on:
– Lawson Test of Scientific Reasoning (Lawson),
– Chemistry Concept Inventory (Mulford and
Robinson),
– Trends in International Mathematics and Science
Study (TIMSS),
– Energy Concept Inventory (Swackhamer et al.)
served as an assessment of student and teacher content
understanding of topics in Physical Science
Gains on PSCI
• gain calculated using the formula developed by Hake1.
g=
(% post-% pre)
100%-% pre
Participants
• Pre-/post-test during 1st summer workshop
Students in subset of participants’ classes
• Baseline established in April 2004
• Given as pre-/post-test in Fall 2004/Spring 2005 and in Fall
2005/Spring 2006
1R.
Hake, “Interactive engagement versus traditional methods: a six-thousand-student survey of mechanics
test data for introductory physics courses,: Am. J. Phys., 65 (5) 418-428 (1997).
Student Performance on PSCI
baseline (N=213)
(% correct)
pre-test (F04)
(N=610)
(% correct)
post-test (S05)
(N=395)
(% correct)
pre-test (F05)
(N=298)
(% correct)
post-test (S06)
(N=262)
(% correct)
31%
30%
31%
30%
36%
•Student performance was essentially the same for the baseline, the
pre-test, and the post-test during the 1st year of implementation.
•A mean post-test score of 36% was achieved at the end of the 2nd year
of implementation. Although this is a small gain, it does indicate some
improvement over what had been achieved in the past. The 2nd year data
focused on the teachers who used the modeling curriculum at a significant level.
Summary
Implementation 1 (F04-S05)
Looking at the entire set of PSCI data from year 1, there is little
evidence for improvement in performance on the diagnostic.
There does seem to be clear improvement in student responses
to question 13, which deals with the concept of area.
Looking at the data for individual sections, one section does
seem to show some significant gain – scores went from 37%
to 47%.
Implementation 2 (F05-S06).
Looking at the entire set of data from year 2, there is a greater
degree of improvement, but the gain is still small.
In one case (section 9), students went from a pre-test score of
33% to a post-test score of 60%, a 39% gain.
There does seem to be clear improvement on a greater number
of individual questions.
Assessing Attitudes
about Science
Student Attitudes
• Attitudes Survey
– consists of twenty-two statements, in which participants
are asked whether they agree or disagree on a five point
Lickert Scale.
– Given as a pre and a post test.
• Statements from the Attitudes survey were obtained
from the:
– Maryland Physics Expectations Survey (MPEX,
University of Maryland PERG),
– the Epistemological Beliefs Assessment for Physical
Science (EBAPS, Elby) and
– the Epistemological Questionnaire (M. Schommer.)
Sample Statements from
the Attitudes Survey
• I developed a better understanding of science
by discussing my ideas with my classmates
and my teachers.
• Learning science makes me change some of
my ideas about how the physical world
works.
Summary of Results
There was little change toward expert-like
views after approximately one year of
modeling instruction.
Researchers have found that attitudes toward science
and science learning science are typically very difficult
to change in the course of a semester or a year.
E. F. Redish, J. M. Saul, and R. N. Steinberg, “Student expectations in
introductory physics,” Am. J. Phys. 66 (3), 212-224 (1998).
Middle School Student Responses
(revised version of instrument)
Results from six sections employing the Modeling Method.
– over 64% of the students stated that this course was
different than other science courses they have taken (Q2).
(Ranged between 64% to 81%.)
– Generally students seemed to feel that discussing ideas
with their classmates and teachers helped them learn the
material (Q11).
(Ranged between 50% to 100%.)
– Students also seemed to feel that they were ready for their
next science course (Q17) and many felt that they were
capable of learning science (Q20).
(In most cases more than half the class responded favorably to these
statements.)
Teacher Self-Assessment
Interviews
Surveys
Interviews with focus group: Summary
• Teachers
– who participated in the focus group were very committed to the
Modeling approach
– were comfortable modifying the instructional materials so that it
better fit their students
– felt that students did have a better understanding - based on
classroom observations and alternate forms of assessment.
• the modeling method created a classroom environment in
which:
– the classroom was more interactive,
– their students began to realize that that they were partly responsible
for their learning
– the teachers recognized that their students were capable of
constructing their own understanding
– the teachers were able to better see where their students were in their
understanding.
Changes in Teaching
Describe how your teaching has changed as a result
of the modeling curriculum.
The two statements that the teachers agreed with
the most:
– my knowledge of concepts in physical science has
improved.
– I now engage my students in more group-work in
the classroom.
Changes in Teaching
Describe how your teaching has changed as a
result of the modeling curriculum.
• Almost all teachers seem to have adopted the
technique of using whiteboards in in their classes to
stimulate group interaction.
• In addition, the use of a standard text book also
decreased. 47% of the teachers stated that they used
the textbook regularly or frequently before being
engaged in modeling and 18% stated regular or
frequent use after.
In general, these results, show a shift toward the approach
emphasized in the modeling curriculum.
Using an Observation Tool
to Assess Teaching
Reform: The RTOP
Purpose of using the RTOPs
• To maintain a standard for observation of
modeling in the classrooms
• To provide a common language as reference
when evaluating the effectiveness of and
implementation of modeling curriculum.
• To provide a standard for evaluation of dialogue
among teachers and mentors (in summer
workshops and during online discussions)
Typical Evaluation Items on the RTOP and how they
informed the evaluation:
Lesson Design and Implementation (1 & 2)
Content (6 & 7)
RTOP example questions
Procedural Knowledge (11- 14)
Other typical RTOP evaluation points
Classroom Culture
Communicative Interactions (16 & 17)
Student/ Teacher Interactions (22-24)
Results from RTOP and RTOP discussion
• Teachers
– improved most on understanding the modeling approach
– struggled with initial implementation due to student resistance
• Students
– use of hands-on manipulatives and experimental design improved
– student ability to communicate results did not improve in most cases.
• The RTOP was only a “snap shot” of teacher
implementation of modeling, yet fostered discussion
and understanding of this method of instruction.
Fascinating results
• Classroom observations showed that fifth
graders’ response to Modeling were very
mature and developed. This occurred in a
public school setting.
• Highest gains in the PSCI were in a
classroom where a teacher (according to
RTOP data) scored the lowest on modeling
skills and implementation of curriculum.
Using Action Research to Assess
the Improvement of Teaching
and Learning Physical Science
Three Action Research Projects
Kushner, K.M. (2006). A comparison of constructivist and traditional
methods on students’ scientific measurement skills. Unpublished
master’s action research study, Cardinal Stritch University,
Milwaukee, Wisconsin.
Gettelman, K. (2006). The use of a modeling method to teach physical
science concepts at the elementary/intermediate level. Unpublished
master’s action research study, Marquette University,
Milwaukee, Wisconsin.
Ferger, M.L. (2007). Traditional vs. modeling instruction of eighth grade
chemistry students. Unpublished master’s action research study,
Cardinal Stritch University, Milwaukee, Wisconsin.
Questions
• How did student understanding of physical science
and related math concepts change?
• How, if at all, did student attitudes toward science
change?
• In what ways, if any, has student engagement n the
scientific process increased – i.e. making hypotheses,
collecting and analyzing data, and forming
conclusions to understand and explain physical
science concepts?
• What is the impact of a modeling curriculum on
student achievement, in comparison to a traditional
science curriculum?
Action Research Designs
Researcher
Kelly
Kushner
Kristi
Gettelman
Megan Ferger
Subjects &
Sample
Population
45 – 8th Grade
Students
30- 5th Grade
Students
100 – Eighth
Grade Students in
4 Science classes
School at which
study took place
midwestern
Midwestern urban
suburban public
public elementary
intermediate school school
midwestern urban
public middle
school
Researcher
Kelly
Kushner
Kristi
Gettelman
Megan
Ferger
Content Taught
Measurement
Physical Science
Chemistry
Teaching
Techniques Used
Modeling vs.
Traditional
Modeling
Modeling vs.
Traditional
Assessment
instruments
implemented
Measurement
Pre and Post
Test
Student Science Attitudes Pre/Post
Survey – all participants
Properties of Matter Pre/PostAssessment – all participants
Classroom Observations and
Reflections
Student Reflections in Science Log –
all participants
Student Interviews – 6 participants
A combined
fact and
critical
thinking pre
and post-test
Findings
•
Students showed greatest gains when they were taught with
a blended approach of traditional and modeling methods
•
Students showed greatest gains in their understanding of
principles of matter and the least in their understanding and
ability to apply advanced measurement principles
•
Student resistance to modeling was a barrier that needed to
be addressed initially; students were too used to being given
right answers rather than thinking for themselves.
• Student attitudes toward science became more positive.
• Student engagement in scientific processes has increased.
What We Have Learned through
Action Research
• Better understanding of the physical science
concepts we are teaching
• Constructivist principles which underlie the
modeling approach to teaching science
Understanding Our Learners
• Common student
misconceptions about
physical science and
how to address those
misconceptions in
teaching
• How changing our
practice can impact
student learning;
• Middle school students
vary greatly in ability to
reason abstractly
Understanding Our Instruction
• A blended model of traditional and
constructivist methods is most effective
• Became more conscious of our own
instructional design and planning and how to
use student work samples and classroom
discussion to guide and inform planning
• Changed our focus from teacher to studentdriven instruction and assessment.
• Applied scientific principles to instruction,
measured benefits to student learning
Where do we go from here?
What will we, as science educators,
take forward from this experience?
Ongoing Action Research
We will continue to use action research
methodology to monitor student learning and
inform our teaching
• Pre/post testing
• Multiple data sources
• Comprehensive data analysis
Continuing the Modeling Method
• Using modeling methods
to teach variety of science
content, rather than just
physical science
• Using whiteboard
presentations in other areas
of curriculum to promote
student-to-student
questioning and
understanding
Extending the Modeling Method
• Sharing findings with colleagues, in order to
improve science instruction in other areas of
school/district
• Connecting physical science and math
instruction, particularly measurement and
geometry strands, to promote deeper student
understanding