Faculty Research & Creativity Forum
October 30, 2008
D. MacIsaac, Physics
J. Zawicki, Earth Sciences and Science Education
L. Gomez, Physics
SUNY Buffalo State College
T. Johnson, Erie 1 BOCES, WNYRIC

validity
Curriculum Standards
•Frameworks
•Syllabi
•Guides
•Blueprints
•Benchmarks correlation
•Objective tests
•Performance assessments
•Portfolios
•Teacher Observations
•Group Activities
•Program Evaluations
Assessment/Evaluation System alignment
•Instructional styles
•Print materials
•Equipment
•Facilities
•Technology
•Community
Instructional Program
False Dichotomies 2





◦ Difficulty (Facility)
◦ Discrimination
◦ Response pattern
1
◦ Student
◦ Instructional (Teacher, School)
◦ Testing
1 NYS Biology Mentor Network
False Dichotomies 3

•You didn’t teach the associated core major understandings.
•You didn’t reinforce the core understandings enough.
•You taught the core content wrong
•Content Knowledge?
•Literacy/Reading
Comprehension?
•Question interpretation Skills?
•Misconception?
–From previous instruction?
–From culture contexts?
–Insufficient reinforcement?
•Effort?
Student
Validity
Instruction
Testing
•Difficulty (Facility) Level?
•Discrimination?
•Placement on exam?
•Visual distraction by nearby
(graphic) items?
•Style of Question?
•Flawed item?
False Dichotomies 4

False Dichotomies 5

10 Earth’s mass is approximately 81 times the mass of the
Moon. If Earth exerts a gravitational force of magnitude F on the Moon, the magnitude of the gravitational force of the Moon on Earth is:
(1) F
(2) F/81
(3) 9F
(4) 81F
Item Difficulty 1
2 3 4
1072 1718 66 317
NR
False Dichotomies 6


False Dichotomies 7

2
Item Difficulty 1
3 0.38
153
2
1220
3
129
4
1597
NR
74
False Dichotomies 8

A horizontal force of 8.0 newtons is used to pull a 20.-newton wooden box moving toward the right along a horizontal, wood surface, as shown.
61. Starting at point P on the diagram in your answer booklet, use a metric ruler and a scale of 1.0 cm = 4.0 N to draw a vector representing the normal force acting on the box. Label the vector F
= 0.49
N
. [1] D
62. Calculate the magnitude of the fricitional force acting on the box.
(Show all work.) [2] D = 0.72
63. Determine the magnitude of the net force acting on the box. [1]
D = 0. 47
64. Determine the mass of the box. [1] D = 0.76
65. Calculate the magnitude of the acceleration of the box. [Show all work.] [2] D = 0.80
False Dichotomies 9



False Dichotomies 10

False Dichotomies 11

False Dichotomies 12

False Dichotomies 13

False Dichotomies 14

False Dichotomies 15

False Dichotomies 16


Teachers often underestimate the critical role played by conceptual physics reasoning which is quite difficult to teach and learn, and focus exclusively on numeric problem solving which is often already taught to students in other classes – like in chemistry and mathematics courses. The conceptual questions on NYSED Regents’
Physics exams are routinely amongst the most difficult items for HS physics students, and a common reaction from students and less successful teachers is that these are “trick questions.” However, these conceptual questions lie at the heart of the physics, and calculating physical quantities that cannot be reliably interpreted by students shows both a weakness in physics teaching and physics learning. Calculation emerges from, is validated and interpreted by conceptual understanding, and conceptual learning is not useful in physics unless it can be used to numerically predict natural phenomena.
Additional information: J. Zawicki <zawickjl@buffalostate.edu>
False Dichotomies

We have observed amongst pre-service and new HS
Physics Teachers a
(aka “conceptual problems”) are that physics problems that do not involve mathematical calculations
than those involving mathematical calculations (aka “numeric problems”). Teachers are very likely to allocate student instruction and learning resources in physics according upon their beliefs. NY HS Physics teachers are driven by the NYSED Regents’ Physics Exam. Hence, we have selected and analyzed several elementary mechanics questions from about 1500 student scores on various NYSED Regents Physics Exams that challenge teacher perceptions of difficulty and numeric vs conceptual content.
False Dichotomies 18

The June 2003 Regent’s Physics exams juxtaposed several numeric and conceptual problems including (on the same page) items #12 and #15.
Item #15, a be the conceptual problem requiring
(of inertia or mass, part of the definition of momentum) that presented extraneous numeric data was later determined to most difficult item on that exam simple definition offering – only about thirty eight percent of about a thousand students were able to answer it correctly.
Item #12, a standard “cookbook-like” problem applying these same definitions and principles (calculating and conserving momentum in a collision) via multiple simple calculations turned out to be in the easiest third of the exam that year, answered correctly by over seventy percent of the students that year.
False Dichotomies 19


A significant fraction of the NYSED Regents physics exam items make use of mass and weight calculations. Students have asked to calculate weight (a force) from mass and apply this information in different problems on every recent exam, often in complex multi-step problems as shown in the example. Students do relatively well on these items.
Periodically the exam directly challenges the student to differentiate between mass and weight.
These items are usually the most difficult items on that offering of the exam.
False Dichotomies 20

NYSED Regents Physics exams prevalently include calculation of inverse square law forces (electrostatic and gravitation), often combined with proportional reasoning and often requiring interpretations of graphic representations of these forces.
When presented as a direct calculational exercise as seen on item 14 from the June 2007 exam, students perform well, despite the fact that this calculation requires significant expertise with scientific notation and calculators.
When asked for simple restatements of these gravitational interactions (the definition of any force), students find these questions about twice as difficult, amongst the most difficult on the exam. See item 12 from the June 2005 exam.
False Dichotomies 21