Formative Assessment Materials for LargeEnrollment Physics Lecture Classes
David E. Meltzer
Department of Physics
University of Washington
www.physicseducation.net
Supported by NSF DUE-0243258, DUE-0311450, PHY-0406724, and PHY-0604703
Real-time In-class Formative Assessment
• The Problem: How can the instructor assess
students’ thinking during class and modify inclass instructional activities accordingly?
• Our Goal: Develop and test materials that both
– provide a basis for in-class instructional activities, and
– assist the instructor in monitoring student thinking,
moment-to-moment…
…in the context of large-enrollment classes
Our Materials:
Carefully sequenced sets of multiple-choice questions
• Emphasize qualitative, conceptual items
• Make heavy use of multiple representations
• Designed to maximize student-instructor
interaction in large classes
• Allow rapid assessment of student learning
• Assist instructors in structuring and guiding
their presentations and instructional activities
Our Materials:
Carefully sequenced sets of multiple-choice questions
• Emphasize qualitative, conceptual items
• Make heavy use of multiple representations
• Allow rapid assessment of student learning
• Assist in structuring and guiding the
presentations and instructional activities
Motivation
Research in physics education suggests that:
• Problem-solving activities with rapid feedback
yield improved learning gains
• Eliciting and addressing common conceptual
difficulties improves learning and retention
Active-Learning Pedagogy
(“Interactive Engagement”)
• problem-solving activities during class time
– student group work
– frequent question-and-answer exchanges
• “guided-inquiry” methodology: guide students with
leading questions, through structured series of
research-based problems dress common learning
Goal: Guide students to “figure things out for
themselves” as much as possibleuide students to
“figure things out for themselves” as much as possible
Key Themes of Research-Based
Instruction
• Emphasize qualitative, non-numerical questions
to reduce unthoughtful “plug and chug.”
• Make extensive use of multiple representations
to deepen understanding.
(Graphs, diagrams, words, simulations, animations, etc.)
• Require students to explain their reasoning
(verbally or in writing) to more clearly expose
their thought processes.
Active Learning in Large Physics Classes
• De-emphasis of lecturing; Instead, ask students to
respond to questions targeted at known difficulties.
• Use of classroom communication systems to obtain
instantaneous feedback from entire class.
• Incorporate cooperative group work using both
multiple-choice and free-response items
Goal: Transform large-class learning environment into “office”
learning environment (i.e., instructor + one or two students)
Active Learning in Large Physics Classes
• De-emphasis of lecturing; Instead, ask students to
respond to questions targeted at known difficulties.
• Use of classroom communication systems to obtain
instantaneous feedback from entire class.
• Incorporate cooperative group work using both
multiple-choice and free-response items
Goal: Transform large-class learning environment into “office”
learning environment (i.e., instructor + one or two students)
“Fully Interactive” Physics Lecture
DEM and K. Manivannan, Am. J. Phys. 70, 639 (2002)
• Use structured sequences of multiple-choice
questions, focused on specific concept: small
conceptual “step size”
• Use student response system to obtain
instantaneous responses from all students
simultaneously (e.g., “flash cards”)
[a variant of Mazur’s “Peer Instruction”]
Results of Assessment
• Learning gains on qualitative problems are
well above national norms for students in
traditional courses.
• Performance on quantitative problems is
comparable to (or slightly better than) that of
students in traditional courses.
• Typical of other research-based instructional
methods
Features of the Interactive Lecture
• High frequency of questioning
• Easy questions used to maintain flow
• Many question variants are possible
• Instructor must be prepared to use diverse
questioning strategies
High frequency of questioning
• Time per question can be as little as 15
seconds, as much as several minutes
– similar to rhythm of one-on-one tutoring
• Maintain small conceptual “step size” between
questions for high-precision feedback on
student understanding.
Curriculum Requirements for Fully
Interactive Lecture
• Many question sequences employing multiple
representations, covering full range of topics
• Free-response worksheets adaptable for use
in lecture hall
• Text reference (“Lecture Notes”) with strong
focus on conceptual and qualitative questions
Workbook for Introductory Physics
(DEM and K.
Manivannan, CD-ROM, Vol. II, preliminary edition, 2002)
Curriculum Requirements for Fully
Interactive Lecture
• Many question sequences employing multiple
representations, covering full range of topics
• Free-response worksheets adaptable for use
in lecture hall
• Text reference (“Lecture Notes”) with strong
focus on conceptual and qualitative questions
Workbook for Introductory Physics
(DEM and K.
Manivannan, CD-ROM, Vol. II, preliminary edition, 2002)
Further development supported by NSF under
“Assessment of Student Achievement” program
Curriculum Requirements for Fully
Interactive Lecture
• Many question sequences employing multiple
representations, covering full range of topics
• Free-response worksheets adaptable for use
in lecture hall
• Text reference (“Lecture Notes”) with strong
focus on conceptual and qualitative questions
Workbook for Introductory Physics
(DEM and K.
Manivannan, CD-ROM, Vol. II, preliminary edition, 2002)
Further development supported by NSF under
“Assessment of Student Achievement” program
Part 1: Table of Contents
Part 2: In-Class Questions and
Worksheets, Chapters 1-8
Part 3: Lecture Notes
Chapter 1: Electric Charges and Forces
Chapter 2: Electric Fields
Chapter 3: Electric Potential Energy
Chapter 4: Electric Potential
Chapter 5: Current and Resistance
Chapter 6: Series Circuits
Chapter 7: Electrical Power
Chapter 8: Parallel Circuits
Chapter 9: Magnetic Forces & Fields
Chapter 10: Magnetic Induction
Chapter 11: Electromagnetic Waves
Chapter 12: Optics
Chapter 13: Photons and Atomic Spectra
Chapter 14: Nuclear Structure and
Radioactivity
Part 4: Additional Worksheets
Chapter 1: Experiments with Sticky Tape
Chapter 2: Electric Fields
Chapters 6 & 8: More Experiments with
Electric Circuits
Chapter 7: Electric Power, Energy Changes
in Circuits
Chapter 8: Circuits Worksheet
Chapter 9: Investigating the Force on a
Current-Carrying Wire
Chapter 9: Magnetism Worksheet
Chapter 9: Magnetic Force
Chapter 9: Torque on a Current Loop in a
Magnetic Field
Chapter 10: Magnetic Induction Activity
Chapter 10: Magnetic Induction Worksheet
Chapter 10: Motional EMF Worksheet
Chapter 9-10: Homework on Magnetism
Chapter 11: Electromagnetic Waves
Worksheet
Chapter 12: Optics Worksheet
Chapter 13: Atomic Physics Worksheet
Chapter 14: Nuclear Physics Worksheet
Part 5: Quizzes
Part 6: Exams and Answers
Part 7: Additional Material
Part 8: “How-to” Articles
Promoting Interactivity in Lecture Classes
Enhancing Active Learning
The Fully Interactive Physics Lecture
Part 9: Flash-Card Masters
Part 10: Video of Class
video
AUTHORS:
David E. Meltzer: Department of
Physics and Astronomy, Iowa State
University, Ames, IA 50011
dem@iastate.edu
Kandiah Manivannan: Department of
Physics, Astronomy, and Materials
Science, Southwest Missouri State
University, Springfield, MO 65804
kam319f@smsu.edu
Overall Tasks
• Develop question sequences for complete
(two-semester) course
• Obtain feedback through in-class use to aid
evaluation of assessment items
• Obtain interview data to validate items
• Do pre- and post-testing with standardized
diagnostics to help monitor effectiveness
Design Strategies
• Not possible to pre-determine all possible
discussion paths
• Knowledge of probable conceptual sticking
points is important: guided by research
• Make use of standard question variants
Many question variants are possible
• Minor alterations to question can generate
provocative change in context.
– add/subtract/change system elements (force,
resistance, etc.)
• Use standard questioning paradigms:
– greater than, less than, equal to
– increase, decrease, remain the same
– left, right, up, down, in, out
Easy questions used to maintain flow
• Easy questions (> 90% correct responses)
build confidence and encourage student
participation.
• If discussion bogs down due to confusion,
can jump start with easier questions.
• Goal is to maintain continuous and productive
discussion with and among students.
Interactive Question Sequence
• Set of closely related questions addressing
diverse aspects of single concept
• Progression from easy to hard questions
• Use multiple representations (diagrams,
words, equations, graphs, etc.)
• Emphasis on qualitative, not quantitative
questions, to reduce “equation-matching”
behavior and promote deeper thinking
“Flash-Card” Questions
“Flash-Card” Questions
Examples of student response data
(Algebra-based general physics at Iowa State University)
Intended to assist instructors at other
institutions in selection and planning in
use of item sequences
#1:
A: 0%
B: 7%
C: 93%
D: 0%
E: 0%
#2:
A: 10%
B: 8%
C: 77%
D: 2%
E: 5%
#7:
A:
B:
C:
D:
E:
2%
3%
3%
83%
9%
#8:
A:
B:
C:
D:
E:
0%
2%
8%
87%
3%
#9:
A:
B:
C:
D:
E:
0%
13%
7%
53%
22%
#10:
A:
B:
C:
D:
E:
67%
20%
9%
2%
0%
Problem “Dissection” Technique
• Decompose complicated problem into
conceptual elements
• Work through problem step by step, with
continual feedback from and interaction with
the students
• May be applied to both qualitative and
quantitative problems
Example: Electrostatic Forces
Problem “Dissection” Technique
• Decompose complicated problem into
conceptual elements
• Work through problem step by step, with
continual feedback from and interaction with
the students
• May be applied to both qualitative and
quantitative problems
Example: Electrostatic Forces
Problem “Dissection” Technique
• Decompose complicated problem into
conceptual elements
• Work through problem step by step, with
continual feedback from and interaction with
the students
• May be applied to both qualitative and
quantitative problems
Example: Electrostatic Forces
Problem “Dissection” Technique
• Decompose complicated problem into
conceptual elements
• Work through problem step by step, with
continual feedback from and interaction with
the students
• May be applied to both qualitative and
quantitative problems
Example: Electrostatic Forces
Problem “Dissection” Technique
• Decompose complicated problem into
conceptual elements
• Work through problem step by step, with
continual feedback from and interaction with
the students
• May be applied to both qualitative and
quantitative problems
Example: Electrostatic Forces
Four charges are arranged on a rectangle as shown in Fig.
1. (q1 = q3 = +10.0 µC and q2 = q4 = -15.0 µC; a = 30 cm
and b = 40 cm.) Find the magnitude and direction of the
resultant electrostatic force on q1.
Question
#1: How many forces (due to electrical
ff
interactions) are acting on charge q1?
(A) 0 (B) 1 (C) 2 (D) 3 (E) 4 (F) Not sure/don’t know
Four charges are arranged on a rectangle as shown in Fig.
1. (q1 = q3 = +10.0 µC and q2 = q4 = -15.0 µC; a = 30 cm
and b = 40 cm.) Find the magnitude and direction of the
resultant electrostatic force on q1.
Question
#1: How many forces (due to electrical
ff
interactions) are acting on charge q1?
(A) 0 (B) 1 (C) 2 (D) 3 (E) 4 (F) Not sure/don’t know
Four charges are arranged on a rectangle as shown in Fig.
1. (q1 = q3 = +10.0 µC and q2 = q4 = -15.0 µC; a = 30 cm
and b = 40 cm.) Find the magnitude and direction of the
resultant electrostatic force on q1.
Question
#1: How many forces (due to electrical
ff
interactions) are acting on charge q1?
(A) 0 (B) 1 (C) 2 (D) 3 (E) 4 (F) Not sure/don’t know
Four charges are arranged on a rectangle as shown in Fig.
1. (q1 = q3 = +10.0 µC and q2 = q4 = -15.0 µC; a = 30 cm
and b = 40 cm.) Find the magnitude and direction of the
resultant electrostatic force on q1.
Question
#1: How many forces (due to electrical
ff
interactions) are acting on charge q1?
(A) 0 (B) 1 (C) 2 (D) 3 (E) 4 (F) Not sure/don’t know
For questions #2-4 refer to Fig. 2 and pick a direction from
the choices A, B, C, D, E, and F.
Question #2: Direction of force on q1 due to q2
Question #3: Direction of force on q1 due to q3
Question #4: Direction of force on q1 due to q4
For questions #2-4 refer to Fig. 2 and pick a direction from
the choices A, B, C, D, E, and F.
Question #2: Direction of force on q1 due to q2
Question #3: Direction of force on q1 due to q3
Question #4: Direction of force on q1 due to q4
For questions #2-4 refer to Fig. 2 and pick a direction from
the choices A, B, C, D, E, and F.
Question #2: Direction of force on q1 due to q2
Question #3: Direction of force on q1 due to q3
Question #4: Direction of force on q1 due to q4
For questions #2-4 refer to Fig. 2 and pick a direction from
the choices A, B, C, D, E, and F.
Question #2: Direction of force on q1 due to q2
Question #3: Direction of force on q1 due to q3
Question #4: Direction of force on q1 due to q4
Let F2, F3, and F4 be the magnitudes of the force on
q1 due to q2, due to q3, and due to q4 respectively.
Question #5. F2 is given by
(A) kq1q2/a2
(B) kq1q2/b2
(C) kq1q2/(a2 + b2)
(D) kq1q2/√(a2 + b2)
(E) None of the above
(F) Not sure/Don’t know
Question #6. F3 is given by
(A) kq1q3/a2
(B) kq1q3/b2
(C) kq1q3/(a2 + b2)
(D) kq1q3/√(a2 + b2)
(E) None of the above
(F) Not sure/Don’t know
Let F2, F3, and F4 be the magnitudes of the force on
q1 due to q2, due to q3, and due to q4 respectively.
Question #5. F2 is given by
(A) kq1q2/a2
(B) kq1q2/b2
(C) kq1q2/(a2 + b2)
(D) kq1q2/√(a2 + b2)
(E) None of the above
(F) Not sure/Don’t know
Question #6. F3 is given by
(A) kq1q3/a2
(B) kq1q3/b2
(C) kq1q3/(a2 + b2)
(D) kq1q3/√(a2 + b2)
(E) None of the above
(F) Not sure/Don’t know
(etc.)
Ongoing Curricular Development
(Projects starting 2003)
• “Formative Assessment Materials for LargeEnrollment Physics Lecture Classes”
Funded through NSF’s “Assessment of Student
Achievement” program
• “Active-Learning Curricular Materials for Fully
Interactive Physics Lectures”
Funded through NSF’s “Course, Curriculum, and Laboratory
Improvement – Adaptation and Implementation” program
Ongoing Curricular Development
(Projects starting 2003)
• “Formative Assessment Materials for LargeEnrollment Physics Lecture Classes”
Funded through NSF’s “Assessment of Student
Achievement” program
• “Active-Learning Curricular Materials for Fully
Interactive Physics Lectures”
Funded through NSF’s “Course, Curriculum, and
Laboratory Improvement – Adaptation and
Implementation” program
Project Phases
• Complete the development of question
sequences for Chaps. 10-14 of Volume II of
Workbook for Introductory Physics
• Acquire baseline data by administering
questions in class with electronic student
response system
• Begin work on question sequences for initial
chapters of Volume I of Workbook
Materials Development
• Carried out by DEM and Ngoc-Loan Nguyen (Iowa
State U. graduate student)
• Created question sequences for electrodynamics,
optics, modern physics, thermodynamics, and
mechanical forces
• Acquired baseline data (at Iowa State University)
for questions on electrostatics
• Currently carrying out editing and additional data
acquisition
Contact:
David E. Meltzer
dem@physicseducation.net
www.physicseducation.net