Assessing Gender Differences in Students'
Understanding of Magnetism using Magnetism
Conceptual Survey
Jing Li
Department of Physics and Astronomy
University of Pittsburgh
Co-Author: Chandralekha Singh
Background
•
Several prior studies on gender effects in college introductory physics
–
–
–
–
–
–
–
•
K. Heller, APS April meeting Talk, Denver, CO, (2009).
M. Lorenzo, C. Crouch and E. Mazur, Am. J. Phys. 74(2), 118, (2006).
S. Pollock, N. Finkelstein and L. Kost, Phys. Rev. ST PER 3, 010107 (2007).
L. Kost et al., Phys. Rev. ST PER 5, 010101 (2009).
P. Kohl, H. Kuo, PERC Proceedings. 1179, 173 (2009).
L. Kost et al., PERC Proceedings, 1179, 177 (2009).
L. Kost et al., PhysRev: ST Phys Ed. Rsrch 6 (2), 020112 (2010) .
Generally observed that in traditionally taught courses
– Males outperform females on FCI/BEMA
– Even if in some courses the final exam grades on traditional exams comparable (not
consistent across various studies)
•
Especially designed courses (e.g., at Harvard course developed by the
Mazur group) can reduce the gender difference
•
Values affirmation and Self-efficacy issues and accumulation of bias of
various kinds over time may play a role in gender differences
– Miyake et al. Science 330, 1234 (2010).
Background
•
Difficult to pinpoint one reason for gender differences between
males and females since issue is very complex
Also, remedies suggested by various groups are different
•
– For example, Z. Hazari, G. Sonnert & P. Sadler (AAPT meeing, Ann Arbor, 2009)
claim that presence of women role models is unlikely to bridge the gap
between male and female performance because female students are unlikely to
identify with the role models.
– Also see, Z. Hazari, R. Tai and P. Sadler, Science Education, 847, 2007.
– Hazari and G. Potvin, E. Journal of Science Education, 10(1), 33, 1-33, 2005.
– Hazari, P. sadler and R. Tai, Phys Teach, 46, 423, 2008.
– They suggest that female students ‘ identity as a scientist and self-efficacy
issues are at the heart of gender difference in science/math
•
Others emphasize the importance of role models
– For example, S. Yennello (AAPT Meeting, Ann Arbor, 2009)
•
Background
•
Here we investigate gender effects in traditionally taught introductory
algebra-based and calculus-based introductory physics on magnetism
concepts using Magnetism Conceptual Survey (MCS) we developed by
administering it as a pre-test and post-test to large classes
•
Many questions in magnetism require the ability to visualize in three
dimensions (force on moving charges or current carrying wires,
magnetic field produced by current carrying wires etc.)
•
Some research suggests that females generally have a better verbal
ability but worse spatial ability than males which can restrict their
reasoning in 3D
– often there is a correlation between students’ spatial ability and their selfconfidence
– D. Law, J. Pellegrino and E. Hunt, Psychological Science, 4(1), 35 (1993).
– 12. M. Beth Casey, R. Nuttall and E. Pezaris, Journal for Research in Mathematics
Education, 32(1), 28 (2001).
•
Development of the MCS
•
Advantage of multiple-choice test
– Objective, easy to grade, results for different groups can be compared
•
Disadvantage is that thought process is not revealed by
test answers alone
– However, research-based multiple-choice tests in which distracter
choices are designed with input from students can be very effective
•
After discussion of the development of Magnetism
Conceptual Survey
– Gender effects
– before and after instruction
– Different types of classes such as algebra-based, calc-based, honors
students
Development of the MCS
• we developed a 30 item multiple-choice test on magnetism (up to
Faraday’s law) to explore the difficulties students have in
– interpreting these concepts
– Correctly identifying & applying concepts in different situations
• To design good distractor choices, during test development students
were
– given free-response questions asking them for reasoning for their responses
– interviewed individually
• The magnetism test was developed by consulting with faculty members
who teach introductory physics
• What students should definitely be able to do
– Three faculty members went over several versions of the test.
– Several introductory students were asked to answer the test questions individually in
interviews.
Administration
• The test was administered as both a pre- and post-test to a large number
of students at University of Pittsburgh.
– three algebra-based classes and eight calculus-based classes.
– Students were taking a traditional algebra-based or calculus-based E&M
course.
• Pre-tests were administered in the first lecture or recitation at the
beginning of each semester.
• Post-tests were administered in the recitations after instructors finished
all concepts on MCS
– Students were asked to work on the test in one class period (40-50 minutes).
• We keep only the students who took both the pre- and post-test except
those in one algebra class in which most students didn’t provide their
names.
– 267 algebra-based students took the pre-test, and 273 students took the
post-test.
– 575 calculus-based students took both the pre- and post-tests.
Reliability of MCS
• Reliability is a measure of the self consistency
of a whole test. It is high if students are not
guessing
• KR-20 for MCS 0.83 for introductory physics
– Good from standards of test design
• MCS administered to 42 graduate student to
benchmark performance
– KR-20 is 0.87
– Average score is 83%
Point Biserial Coefficient
• A measure of consistency of a single test item with the whole
test.
• The desired value should be greater than 0.2.
• Point biserial coefficient for MCS is 0.42.
Item Difficulty
• A measure of the difficulty of a single test question. It is
calculated by taking the ratio of the number correct responses
on the question to the total number of students who
attempted the question.
• The desired value is over 0.3. Average for MCS is 0.46.
Item Discrimination Index
• A measure of the discriminatory power of each item in a test.
• Can be calculated as
N N
D
H
N /2
L
• The desired value should be greater than 0.3.
• Average item discrimination for MCS is 0.33.
Assessing Gender Difference in the
Performance in Algebra-based Courses
• For algebra-based course, there is no significant difference
between men and women on the pre-test and post-test.
Algebra based Pre-test
Group
Number
Mean
S.D.
Sig.(2-tailed)
Men
91
7.3
2.51
0.940
Women
106
7.1
2.36
Algebra based Post-test
Group
Number
Mean
S.D.
Sig.(2-tailed)
Men
110
13.2
5.20
0.360
Women
121
12.3
5.57
Assessing Gender Difference in the
Performance in Calculus-based Courses
• For calculus-based course, men and women don’t have
significant different on the pre-test while men outperformed
women on the post-test.
Calculus based Pre-test
Group
Number
Mean
S.D.
Sig.(2-tailed)
Men
403
8.5
3.40
0.490
Women
168
7.8
3.05
Calculus based Post-test
Group
Number
Mean
S.D.
Sig.(2-tailed)
Men
403
15.3
6.20
0.019
Women
168
13.0
5.38
Assessing Gender Difference in the
Performance of the Honors Students
• For honour students, women do not perform as well
on the post-test.
Honored students
Group
Number
Mean
S.D.
Sig.(2-tailed)
Men
75
17.4
5.89
0.030
Women
20
14.1
6.21
Gender Performance on individual
Questions
• In algebra-based classes, males outperformed
females on 20 questions
• 4 questions have difference more than 10%.
• Females outperformed males more than 10% only on
1 question.
• In calculus-based classes, males outperformed
females on 28 questions
• 9 questions have difference more than 10%.
• Female s only performed slightly better on the other
two questions.
Possible Explanations on Gender Difference
• Other research using FCI/BEMA have shown gender difference in
traditional introductory physics
• In addition, to societal factors and accumulated bias discussed in
earlier studies
• In magnetism, most questions require ability to visualize in 3D
– figure out the directions of magnetic field
Lorentz force.
• Some prior research suggests that females generally worse spatial
ability than males
– restrict their reasoning in 3D
– there is a correlation between students’ spatial ability and their self-confidence
Algebra-based vs. Calculus-based course
• Pre-test were given at the beginning of the course
– There is a statistical difference
Group
Number
Mean
S.D.
Sig. (2tailed)
algebra
267
7.2
2.48
0.000
calculus
575
8.3
3.31
• Post-test were given at the end of the course
– Statistically significant difference
• Calculus-based courses do better than algebra-based courses
Group
Number
Mean
S.D.
Sig. (2-tailed)
algebra
273
12.2
5.34
0.010
calculus
575
14.6
6.05
Algebra-based vs. Calculus-based course
Why should there be a difference between
algebra-based and calc-based students when
conceptual questions?
Possible explanations
• Calc-based students have better Math skills and
Scientific reasoning skills
• Calc-based students are less likely to have cognitive
overload while learning
• Calc-based students are more likely to build a robust
knowledge structure
group
Num
ber
Mean
S.D.
Sig. (2tailed)
group
Numb Mean
er
Alge
108
7.43
2.36
0.042
Alge
213
Calc
284
8.24
3.89
Calc
241
S.D.
Sig. (2tailed)
13.45
5.34
0.000
15.87
6.31