Improving on the Recitation Section:
Tutorials in Introductory Physics
Wed Brown Bag
SJP Fa '05
What are Tutorials?
•
•
•
•
Research-based
Student-centered
“Elicit-confront-resolve”
Process (discussion, consensus)
+ Sense-making
The Tutorial space
(basement, across from Jerry Leigh)
Tutorials at CU
• Used in 1110 & 1120, twice each
(+ 1120, now)
• Part of successful courses
=> motivation to continue
• Today: some data
Continue support?
Resources
• Space, material, training, personnel
Ongoing support necessary.
• “Learning Assistants” (LA’s) - undergrad
STEM majors (interested in teaching)
Courses studied so far
Recitation
Homework
Text:
Tutorials
Tutorials
Knight workbook
Trad recitations
CAPA
CAPA
MP
MP
Knight
Knight
Tutorials
Tutorials
CAPA
CAPA
• Phys 1110
–
–
–
–
Fa 03
Sp 04
Fa 04
Sp 05
HRW
HRW
• Phys 1120
–Fa 04
–Sp 05
HRW
Knight
pre/post measures
•
•
•
•
Phys 1110: FMCE
Phys 1120: BEMA
common exam questions
formal + informal survey questions
Phys 1110: Distinguishing features
1: “Tut” (Sp04) Tutorials + CAPA
2: “Workbook” (Fa04) Small groups/Knight
(+ Mastering Physics)
3: “(More) Trad” (Sp05) Mostly traditional
recit’s (Otherwise much like "2" above)
(different instructors, semesters …)
1110 summary - up front!
• Tutorial courses : strongest results on
learning gains and attitude surveys
• Middle course (“2”) (research-based text
and hw, clickers + small-group
recitations, not Tutorials): good results.
• Last course (like “2” except recitations):
gains solid, lowest of three.
Phys 1110 Pretest
FMCE (Force and Motion concept evaluation)
A nationally validated conceptual exam, like the FCI (but harder)
Comparing FMCE pretest
Matched, valid data only. (Eliminate pre>93%)
Ave
40
35
% of students
30
25
20
15
10
5
0
0
9
18
27
36
45
55
64
73
82
91
100
Score (%)
1PreS0 4 (28 % +/-1)
2PreF04 (3 4%)
Pre ave = 28%(1) / 34%(2) / 28%(3)
Spring (2) higher (fall term)
3PreS0 5 (28 % )
Phys 1110 Posttest
Comparing FMCE posttest
Ave
40
35
% of students
30
25
20
15
10
5
0
0
9
18
27
36
45
55
64
73
82
91
100
Score (%)
1Pre S04 (7 4% +/-1)
2Pre F04 (69%)
3Pre S05 (5 8% +/- 2% )
Post ave = 74%(1) / 69%(2) / 58%(3)
Phys 1110 normalized gains
distribution of g: compared
% of students
30
25
20
15
SP04
10
5
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
normalized gain
gain(1)= .66 +/-.02
Phys 1110 normalized gains
distribution of g: compared
% of students
30
25
20
SP04
15
FA04
10
5
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
normalized gain
gain(1)= .66 +/-.02 g(2)= .585 +/-.02
7.5 points lower => (more than half a letter grade)
Phys 1110 normalized gains
distribution of g: compared
% of students
30
25
20
SP04
15
SP05
10
5
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
normalized gain
gain(1)= .66 +/-.02 g(2)= .585 +/-.02 g(3)= .45 +/-.02
(trad recit.) => significantly lower gains. (still, double nat’l standards!)
Impact on different pretest populations:
"low starters" pretest <=12.5%
nor malized gain for low pretest
0.6
0.5
g
0.4
0.3
0.2
0.1
0
S04 (2 3%)
F04 (20%)
S05 (2 2%)
Semester (%
lass in
(% of
of cclass
inthis
this
pool)
prete st <= 12.5%
Course
(1)
(2)
(3)
Impact on different pretest populations:
"high starters" 50<pre<93%
normaliz ed gain for high pretest
0.9
0.8
0.7
0.6
g
0.5
0.4
0.3
0.2
0.1
0
S04 (13 %)
F0 4 (22%)
S05 (14 %)
Semester (%
clas s in
(% of class
in this
this
pool)
pretes t >50%
Course
(1)
(2)
(3)
Beyond the FMCE: Exam comparisons
• #2 (Knight workbooks/small groups):
34 common exam q’s
(17 are “tutor. materials”, 9 are “quant/trad”)
• #3 (Trad recitations):
30 common q’s
(12 are “tutor. materials” 6 are “quant/trad”)
Beyond the FMCE: Exam comparisons
( Tutorial score - other course score)
All q’ Tut mater. quant/trad
(2) Workbook: +6
+7
+6
(3) Trad:
+10
+14
+10
(All ± 2)
N.B. 12 points is roughly 1 letter grade.
Tutorial courses: significantly better exam scores:
both conceptual and quantitative/traditional.
Other data
• Replication study (compare with UW)
• CLASS (attitudes/beliefs)
• Surveys (did you like, did it help?)
All favorable (or neutral) for tutorials,
ask for details!
1110 Summary
• Compare Tut-based with “workbook/small group”
 measurable diffs (FMCE, exams, CLASS)
• Compare Tut-based with “more trad rec”
 significant diffs.
• Tutorials only one effect.
(Instructors, course structure …!?)
• But in 1120, changing instructors + text
=> no impact
Phys 1120:
Tut1 (Fa04) and Tut2 (Sp05)
Attempt at replication.
Main differences: Tut2 has…
• different instructor
• different semester
• different textbook
• follows up 1110 without Tutorials
• no long answer on exams
Summary (up front!)
• Despite changes in course elements,
we replicated Tutorials + basic course
structure.
• Result: identical (high) learning gains.
(Final score matches our juniors.
Hard exam!)
1120 BEMA pre/post
% of students
BEMA (matched) (CU scoring) Fa04
20
18
16
14
12
10
8
6
4
2
0
0
6
12
18
24
30
36
42
48
55
61
67
Score (%) (CU scoring)
PreF04
BEMA = “Brief E&M Assessment”,
F04 (N=319) Pretest ave 26%
73
79
85
91
97
1120 BEMA pre/post
% of students
BEMA (matched) (CU scoring)
Compare Fa04 and Sp05
20
18
16
14
12
10
8
6
4
2
0
0
6
12
18
24
30
36
42
48
55
61
67
73
79
85
91
97
Score (%) (CU scoring)
PreF04
F04 (N=319) Pretest: 26%
PreS05
S05 (N=232):
27%
1120 BEMA pre/post
% of students
BEMA (matched) (CU scoring)
Compare Fa04 and Sp05
20
18
16
14
12
10
8
6
4
2
0
0
6
12
18
24
30
36
42
48
55
61
67
73
79
85
91
97
Score (%) (CU scoring)
PostF04
F04 (N=319) 26% -> 59%,
g(ave, F04) = .44+/- .01
PostS05
S05 (N=232) 27% -> 59%
g(ave, S05)=.43+/- .01
BEMA (matched) (CU scoring) Fa04
g(ave)=.44, N=331
1120 E&M pre/post
CU upp LA 1120
div post pre post
50
LA
TA
post pre
TA
post
45
# of students
40
35
30
25
20
15
10
5
0
0
6
12
18
24
30
36
42
48
54
60
66
72
78
84
90
Score (%) (CU scoring)
Pre (ave=26)
Post (ave=59)
NCSU trad
post
NCSU honor
& CMU trad
post
CMU M&I
post
CMU upp
div post
96
Other data
•
•
•
•
Common exam questions (no change)
Replication study (compare with UW)
CLASS (attitudes/beliefs)
Surveys (did you like, did it help?)
All results similar for both semesters!
Affect: survey results
1120 (Tut2) followed 1110 using Knight
“workbook” small-group recitations.
Asked at end:
Which is better, Tut or trad rec? 39% vs 40%
Which would you enjoy more? 39% vs 39%
1120 mini-summary
Different instructors, text, exam structures,
semesters… same Tutorials and Conceptests:
no sig diff on BEMA, CLASS, exams.
Validated survey scores high.
Slight differences in surveys:
~neutral student satisfaction.
No disasters, room to improve
Bottom line
Tutorials successful, productive course elements.
• Cost:$1500/LA/semester * (6-8 LA’s per course)
• Need 1.5 hr/week training session (TA’s too!)
• Benefits: Some LA’s => K-12 cert.
(+ their learning gains very high)
• TA’s exposed to research-based learning.
Recommendation
• We should continue implementing
Tutorials (and collecting data)
• Need support for LA’s, and training
infrastructure
(=> faculty or experienced grad student
assigned to teach the TA’s/LA’s)
Questions, Discussion!
THE END
Replication
UW
(no Tut)
UW
CU
(with Tut) (with Tut)
Atwood:
tension
25
50
55
Identify
Newton III
partners
15
70
70
McDermott et al., AJP 62, 1994
Replication
UW
(no Tut)
UW
CU
(with Tut) (with Tut)
Atwood:
tension
25
50
55
Identify
Newton III
partners
15
70
70
“Beliefs” survey: CLASS pre/post
1110 Overall CLASS
75
% favorable
1-Tut
2-Workbook
3-Trad
55
12
22
% unfavorable
CLASS pre/post (full scale)
1110 Overall CLASS
100
% favorable
80
60
40
20
0
0
20
40
60
80
100
% unfavorable
Tutorial-based course: no shift
Two others: small but statistically significant declines
CLASS pre/post
1120 Overall CLASS
% favorable
70
Sp05
Tut2
Tut1
60
10
15
20
% unfavorable
pre- to post shifts (attitudes and beliefs survey).
Slightly negative (!), though it’s usually worse.
CLASS pre/post
1120 Overall CLASS
100
% favorable
80
60
40
20
0
0
20
40
60
% unfavorable
(Shown on full 0-100 scales)
80
100
Affect: survey results 1110
(Likert scale 1= “no” to 5=“yes”)
• Did you like the Tutorials?
2.41 +/- .05
(62% neg, 21% pos)
• Did the Tutorial help you learn Physics in 1110?
(20% neg, 60% pos)
3.53 +/- .05
Affect: survey results 1120
Likert scale 1-5: “very bad” to “very good”)
• How do you feel about the use of Tutorials in Phys
1120?
Tut1: 3.32 +/- .07
(33% neg, 53% pos)
Tut2: 2.96 +/- .09
(46% neg, 43% pos)
• How much help was the Tutorial?
Tut1: 3.33 +/- .06
Tut2: 2.90 +/- .08
F
C
I
at
C
U
Force Concept Inventory (FCI)
red = trad, blue = interactive engagement
<g> = post-pre
100-pre
Fa98
Fa03/Sp04
R. Hake, ”…A six-thousand-student survey…” AJP 66, 64-74 (‘98).
Beyond the BEMA - exam q’s
• 31 common exam q’s
(13 explicitly “tutorial material”,
rest => other topics)
(Fa04-Sp05)
All
Tut material only
-1.9% ± 2%
-1.8%
~75%
~40%
* Mazur, E. Peer Instruction, Prentice Hall