Transforming tertiary education: overview
of an AKO Aotearoa project involving a
large 100-level Cell Biology class.
Rosie Bradshaw, Maggie Hartnett, Gemma Cartwright,
Natalie Burr, Ewen Cameron, Ben Kennedy & Zoe Jordens
Applying the Carl Wieman Science Education Initiative (CWSEI) Model
within the New Zealand Tertiary Environment
162101 Biology of Cells
600
Student numbers
500
400
PN
ALB
300
200
100
0
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
Year
BVSc
BSc
BMLS
B Vet Tech
B Food Tech
B Eng
School
Other
Students have diverse
aims, background
and abilities
The team
• Mixed team- Education specialists, motivated science
lecturers and research assistants.
• Six science classes: four in Geology at Canterbury
and two in biology at Massey
The aim
• Improve student engagement and learning by:
- implementing ‘interventions’
- measuring the effects on engagement & learning
based on
The method
Interventions:
• Improved learning goals and alignment
• Emphasis on relevance
• Interaction in lectures: activities and ‘clickers’
Measuring effects:
• classroom observations
• attitude survey
• ‘knowledge’ survey
engagement
learning
The cell shown here is a human diploid cell destined to
undergo meiosis to make an egg cell. The cell is heterozygous
for genes A and D. Assuming no crossing over occurs, choose
the diagram that best represents what this cell would look like
in metaphase of Meiosis I.
A D
A a
B A aD d
A
a
C
d
D d
A A
a
D
a
D D
d
d
E
A Aa
a
D Dd
d
The method
162101 Biology of Cells
semester 1 2011
PRE
POST
1
2
surveys
3
162212 Microbial World
semester 2 2010
PRE
1
2
surveys
semester 2 2011
POST
1
2
surveys
Results: Classroom observations
• Observe lecture material and teaching style
• Measure students engagement through observation
Results: Classroom observations
• Observe lecture material and teaching style
• Measure students engagement through observation
10.00
9.00
8.00
7.00
6.00
5.00
4.00
3.00
2.00
1.00
0.00
Pre-intervention
Post-intervention
Results: Classroom observations
Observation 2
Tour of the Cell Lecture: Pre-Intervention
Student Engagement (out of 10)
10
9
8
enthusiasm
7
Skimmed
content
6
5
4
Movie
near
end of
lecture
Overview
3
2
1
0
0
3
6
7
11
14
16
20
Time (mins)
24
27
30
35
40
44
47
Results: Classroom observations
Link
to lab
Student Engagement (out of 10)
10
9
Question
Question
Analogy
8
7
Learning
objectives
6
5
4
Drawing
activity
3
Analogy
Lecture
Overview
Lecture Lecture
Next session
and vote
whether to
persevere with
clickers
Observation 14
2
Prokaryotic gene expression lecture (post-intervention)
1
0
0
10
15
19
23
24
27
Time (min)
30
32
38
41
42
Results: Attitude survey
Significant shifts compared to ‘expert-like attitudes’ (for all
of the example Qs ‘strongly agree’ is expert-like and therefore favourable)
Category
(4 of 7 shown)
Favourable
Unfavourable
Example of question
(8 – 10 Qs in each category)
Real World
Connection
ns
To understand biology, I sometimes think
about my personal experiences and relate
them to the topic being analyzed.
Enjoyment
ns
If I had plenty of time, I would take a biology
class outside of my major requirements just
for fun.
Problemsolving effort
ns
I actively try to relate what is presented in
biology to what I have learnt in other
courses.
Reasoning
ns
When not pressed for time, I will continue to
work on a biology problem until I understand
why something works the way it does.
shifts in attitude over the whole semester :
ns not significant;
got worse (less ‘favourable’);
got better (less ‘unfavourable’)
Results: Knowledge survey
semester 1 2011
PRE
POST
1
2
3
Questions
based on:
Survey 1
Survey 2
Survey 3
Mean gain
PRE
10 Qs
21.4 ±
14.3
32.7 ± 16.0 39.3 ± 18.5
11.3 ± 16.1
POST
10 Qs
20.9 ±
15.2
27.1 ± 14.6 41.6 ± 19.1
14.5 ± 19.8
Values are % test scores: mean ± SD for 161 matched surveys
(questions were designed to be difficult!)
Two-tailed paired T-test of significant difference between pre and post mean gains shows
no significant difference (P = 0.103)
Results: Knowledge survey
Questions
based on:
Mean %
gain
final exam
>80%
Mean %
gain
final exam
60-79%
Mean %
gain
final exam
40-59%
Mean %
gain
final exam
20-39%
Mean %
gain
final exam
<20%
Mean %
gain
ALL
students
PRE
10 Qs
20.0
15.8
8.6
1.7
-1.7
11.3
POST
10 Qs
32.9
15.9
13.6
2.2
1.7
14.5
P (paired
T-test) (n)
0.007
(14)
0.964
(64)
0.424
(59)
0.941
(18)
0.788
(6)
0.103
(161)
Values are means of matched % test scores
Conclusions:
Most of the ‘post’ (interventions) gain was in the top student group
More needs to be done to facilitate learning in other groups!
BUT
- there are many limitations to this survey.
- deleting toughest 2 Qs from each survey only 40-59% show sig gain
But what do students think?
Clicker-vote at end of 162101 2011:
To assist my learning in lectures, I would prefer:
A) 'Traditional style' information delivery lectures
B) Mixture of clickers/interactive questions and 'traditional
style', with some recommended reading prior to lecture.
C) Completely interactive (clickers/activities/discussion) with
mandatory pre-reading and on-line pre-lecture quiz.
Results: A 5%, B 70%, C 25% (n=207)
Conclusions
No increase shown in student engagement
BUT - engagement declines over the semester
- positive feedback on use of clickers
Mixture of attitude shifts
BUT - some attitudes change over the semester anyway (eg. enjoyment)
Significant increase in knowledge gain after interventions
BUT - only with top performing students (final exam >80%)
- mid students (40-59% FE) gained when toughest Qs removed.
So – was it worth it?
Overall I think my teaching has improved
- Better learning goals, etc
- Improved style of interactive questions
& many ideas of how to improve more…
Acknowledgements
Zoe Jordens – Massey Project Leader
Maggie Hartnett – Education Consultant
Gemma Cartwright, Natalie Burr – Research Assistants
Ewen Cameron – T&L director, MUHEC approval
Ben Kennedy - Canterbury Uni. and overall project leader
Applying the Carl Wieman Science Education Initiative
(CWSEI) Model within the New Zealand Tertiary Environment