A Sample of Best Practices in
University Lower Division Science
Education
CSE Brownbag
18 October
Dan Bernstein, University of Kansas
djb@ku.edu
Overview of session
• Uses of class time
• Capturing out of class time
• Alternative course designs
• Discussion of implementation
• costs and benefits
• cultural context
• recommendations
• Disclaimer
Using class time
• Pause for problems /
interaction
• Mazur is the poster child
• Survey of KU clicker users
– Attendance and pop quizzes
– Check for understanding 3rd
– no plans for how to proceed
• Pollock argued that main
benefit is collaboration during
breaks
– “Stop learning and listen to me”
• Could be routine feature of
classes
Group work tricky in large classes
•
•
•
•
•
•
•
•
Managing groups requires planning
One KU professor takes on Budig 120
Tim Shaftel (Business) inserts group days
Random assignment to fours w/ warmup
Works a problem on the big screen
Breaks out for comments and suggestions
Roams the room for solutions
Consider the video
Tutorials - U of Washington PEG
• Lillian McDermott and colleagues
• Crafted generative problem tutorials
• Intended to replace lecture/problem
sessions [TA doing the problem]
• Active engagement in figuring
conceptual features of physics
• Consider these examples
Collisions in 2-D
Progressive questions per set up
Focus on explanations
More challenging particulars
Magnetism -- with magnets
Progressively complex
Steve Pollock, Physics
University of Colorado
• Replaced typical discussion for
Intro to Physics with Tutorials.
• Result: very high learning gains,
by national standards. (“The final
score matches what our junior
physics majors get on this hard
exam!”)
Colorado -- BEMA pretest
% of students
BEMA (matched) (CU scoring) Fa04
20
18
16
14
12
10
8
6
4
2
0
0
6
12
18
24
30
36
42
48
55
61
67
73
79
85
91
97
Score (%) (CU scoring)
PreF04
BEMA = “Brief E&M Assessment”, validated
research-based survey of Conceptual
elements of E&M. Blue data above is F04
(N=319) Pretest ave 26%
BEMA post -- Comparable to Grad Students
% of students
BEMA (matched) (CU scoring)
Compare Fa04 and Sp05
20
18
16
14
12
10
8
6
4
2
0
0
6
12
18
24
30
36
42
48
55
61
67
73
79
85
91
97
Score (%) (CU scoring)
PostF04
F04 (N=319) 26% -> 59%,
PostS05
S05 (N=232) 27% -> 59%
“Posttest results yield an impressive replication
for two semesters High by nat’l standards
(typical trad courses, post score = 30-40% !)”
Pre/post FMCE (Sp04)
# of students
60
Pre
50
Post
40
30
20
10
0
0
12
24
36 48
61
Score (%)
73
85
97
This is their research area
Inquiry laboratories
• Related to the tutorials -constructivist model of understanding
• Taken to full hands on laboratory
• Joe Heppert, Jim Ellis, Jan Robinson
• Engage students in process
• Embedded, inductive, open-ended
High End -- Studio Physics
•
•
•
•
•
Hands on discovery in place of lecture
Reorganize even very large classes
Two hour blocks of time
Measure conventional and conceptual skills
Taking inquiry lab, constructivist model to
the whole experience
• Robert Beichner, NC State, one example
Teaching space very different
Conventional exam questions
MC items - Studio v. three lecturers
Studio => comparable problem sets
Failure ratio: Lecture/Studio
FCI gain - Highly replicable
Semester gain by class rank
Outside of Class Time
• Some use technology
– Center for Academic
Transformation
• Others based on peers
– Community building
– Meta cognitive
coaching
Carol Twigg invested Pew funds
• Re-gifted the money for course redesign
• Focused on technology as tool
• Emphasized saving money through efficient
non-human or lower cost human delivery
• Committed to evaluation by learning and
completion rates
• Increased success and/or difficulty of course
• Tracked learning downstream in curriculum
• Decreased rates of D, F, and Withdrawal
Carnegie Mellon -- Statistics
• Created StatTutor program
• Open-ended intelligent tutoring software
– Gives feedback on individual paths
– Focuses on decision making en route
• Aimed for high levels of skill not previously
attainable
• 22% increase in scores
• Critical skill is selecting appropriate
statistics to use
High rates of success
• Replicated in two course offerings (N>400)
• Selection error rates dropped from ~9 to <1
• Two skills not attempted before reached
70% correct
Ohio State University - Statistics
• Buffet of options for >3000 students / year
• Discovery laboratories, small groups, small
lectures, training modules, video reviews
• All take common examinations
• Learning was greater than comparable
daytime lecture based course
• Greatly enhance retention of students
• Fewer W’s, F’s, and I’s
• Modular credit (1-5), reducing full retakes
Tutorial out performed day class
• Large class equaled smaller night class
• Fewest failures
• Maintained large enrollment
Penn State - Statistics
• Reduced lectures from 3 to 1 per week
• Replaced with computer lab time
– Computer mediated workshops
– Extended practice in computerized testing
•
•
•
•
•
Lecture: Exam pre-post was 50% => 60%
Redesign: Exam pre-post was 50% => 68%
Selection of correct tool: 78% => 87%
DFW rate: 12% => 10%
2200 students per year
University of Iowa - Chemistry
• 1300 students / year
• Pressure from Engineering and Pharmacy
• Fewer lectures, modular content, active
participation, computer simulations
• Inquiry based laboratories
• No difference on common exam items
• Am Chem Soc exams: 58 => 65, 52 => 61
• DFW: 24-30% => 13%
• DF: 16% => 9%; W: 9% => 4%
U Mass - General Biology
•
•
•
•
•
•
•
•
•
700 students / semester
Lectures: 3 => 2, add review session
Inquiry lab already in place
Interactive class technology, online quizzes
Peer tutoring and supplemental instruction
Use ClassTalk network for students
Exams: 61% => 73% correct
Questions: 23% => 67% required reasoning
DF: 37% => 32%
Peer led workshop groups
•
•
•
•
•
Northwestern University Biology course
Based on legendary work of Uri Treisman
Peers prepared as facilitators at UT Austin
Led group problem solving 2 hr / week
Majority students outperformed controls
– Steady improvement across exams
• Minority students outperformed controls
– Improvement noted on last exam
– Historic controls show decline over course
– 3rd exam exceeds majority controls
Wendi Born @ CTE 7 November
Supplemental Instruction
• Peer led sessions with trained facilitators
• Part content and part meta cognition
– Study skills
– Learning about learning
• Designed at UMKC by Deanna Martin
– Address high failure rate by minorities in
professional programs
– Identifies at risk courses, not at risk students
• Lani Guinier on the canary
Key Characteristics
• All students invited, not targeting weakest
• Always with faculty cooperation
• Sessions begin right away
– Not associated with having problems
• Minority students:
– SI participants have 2.02 GPA in courses
– Non SI participants have 1.55 in same courses
• DFW rate:
– Non SI at 43%, SI at 36%
Huge international following
Nebraska - Intro to Chemistry
Non SI had more
than double the
failure rate
83% passed with
SI, 57% without
“Universal” aid, like Studio Physics
[Universal] Design for Success
• Presume students
can learn
• Discount need to
sort or differentiate
• Maximize overall
course performance
Benjamin Bloom promoted mastery
• Based on practice
and feedback
• Divide course into
many smaller units
• Take examinations
and get results
• Require taking exam
again until high score
• IFF 95% correct =>
study next unit
Fred Keller promoted mastery
• IFF 95% correct =>
study next unit
• Course grade is number
of units passed
• No penalty for repeating
and learning
• All who pass 12 units =>
grade of A
• Do A work on 10 units
=> grade of C
Also taught conventional lecture
• Mastery Class
• 95% contingency
• No penalty for
learning
• Immediate
feedback
• No lecture required
• Lecture class
• Same exam
questions
• Two attempts per
test
• In class feedback
• No contingency
Total amount learned
• Nearly twice as many at the high end of learning
• Virtually no one failed to learn
• Maximized learning for many students
Showed in amount and
accuracy
• Many more questions
answered
• Took 12 15-item tests
• Lecture was three tests
of 20 items each
• Certify more learning
• Overall percent correct
also higher
•
•
•
•
•
No magic -- students studied
better
They put in more time to
their learning
There was more work
asked for by the course
Note that they report
doing the reading more
Preparation for class is
key issue (later also)
Guideline in US -- 2
hours outside for every 1
hour in class
Major meta analysis of 100 studies
•
•
•
•
•
Kulik, Kulik, & Bangert-Drowns
Consistently more learning
More time on task
Greater retention over time
Lower completion rates when used
without deadlines and incentives
Placement and Prerequisites
• Variation on the same theme
• Languages require competence
• Tracking skill downstream in the
curriculum
• Using mastery criteria for foundation
courses
• Requires some coordination within and
between units
• Could benefit from tutorials and SI
Marginal gains not clear
• Are these effects additive?
• Maybe they all help the same students in the
same way
Ernest Boyer:
The work of the scholar
remains incomplete
until it is understood and used
by others.
Challenges on teaching science
• Do we really want success? Grade inflation?
• How do we handle the coverage/depth issue?
• What about the resources?
– Space, funds, faculty time
• Students should also be responsible
• Are these technologies transferable/robust?
• What about bureaucratic hurdles?
– Remedial courses/tutorials, Undergrad TAs
– Semester based credit and tuition
Your Insights?
http://www.cte.ku.edu
Three functions of grading
•
•
•
•
•
Certification of learning
Motivation for learning
Differentiation among learners
Each has a legitimate purpose
No one system does all well
Variability in conventional course
•
•
•
•
Students learn at different rates
When course ends, fast learners get best results
Very good at identifying fast learners (differentiate)
Less good at motivating for more work
Variability in a mastery course
•
•
•
•
Everyone learns until material is mastered
Reward is for work; subjective probability of success
Very good at certification of learning
Provides incentive for studying, no penalty if slow
When is mastery the right
approach?
• Foundation courses -- want knowledge
• Programs in which rate of learning is not a
criterion for success
• Situations in which performance will not be
timed
• Professions in which high skill is expected
• Why tolerate ineffective teaching?
• If we don’t care or think it can not be learned
by all
How much can a student
learn?
•
•
•
•
•
•
•
Boundaries are time, effort, and capacity
Time and capacity are fixed
Your leverage into learning is effort
Organize a system that allows extra work
Honor that work when it succeeds
May lose some identification of capacity
Greatly expand the amount of learning
Scholarship Assessed (1997)
• All forms of scholarship
include:
– Clear goals
– Adequate preparation
– Appropriate methods
– Significant results
– Reflective critique
– Effective presentation
Glassick, Huber, & Maeroff
Communities of inquiry on learning
• Being very public with teaching in same
sense of a center for research
• Faculty need another lens to complement
student voice, converging measures
• Have an external community that values
this work
• Stresses our existing skills at intellectual
inquiry as basis of exploration
Building a community to discuss
ideas about teaching
• Workshops and seminars for faculty
members
• Straightforward process of peer interaction
• Exchange ideas around three themes
• Provide resources for exploration
• Written product of thinking and planning
Collaborative Working Seminars
• Discussion of
shared issues with
colleagues
• Time for reading and
searching
• Targets for writing
and sharing
• Intentional plan is
the product