CETL-impact : case studies from
S104 Exploring Science
Sally Jordan, Diane Butler and Paul Hatherly
3rd Open CETL Conference, 24th September 2008
Enabling student planning and
preparedness: use of diagnostic
quizzes and the Maths Skills ebook
Sally Jordan
3rd Open CETL Conference
The problem…
• Making sure that students are on the ‘right’ course and
sufficiently prepared for study is of crucial importance for
retention.
• S104 is a 60 point course: we want to make sure that
students have sufficient time available for studying it.
• S104 assumes some basic mathematical skills, taught in
S154 Science Starts Here and elsewhere. Where
appropriate, we want students to study S154 before S104.
• Other students may be generally well prepared, but rusty
in a particular skill assumed by S104.
• So we use an online diagnostic quiz ‘Are you ready for
level 1 science?’ and the Maths Skills ebook.
Are you ready for Level 1
Science?
• A successor to the printed ‘Are you ready for…’
diagnostic quizzes;
• Three linked quizzes: online, interactive;
• Designed to help students assess their preparedness for
S104, S154 and other level 1 science courses;
• The S104 quiz makes extensive use of iCMA questions
written for S154 (with different data-sets).
AYRF
Are you ready for Level 1
Science? : Evaluation
• 50,000 people have accessed the quizzes since April
2007; 14,000 have accessed ‘Are you ready for S104?’
• Students have used the quizzes to
Confirm choice of course;
Decide between two or more courses;
Alter choice of course;
Identify small areas of weakness.
• Of S104 students who used the quizzes, 90.3% found
them useful or very useful.
• But 50.4% of S104 students surveyed didn’t use the
quizzes before starting S104.
The Maths Skills ebook
• Developed as a piCETL resource, initially for Science
Short Course students.
• Available on the S104 course website and the new
Studying Science website, to enable students to access
it before the S104 site opens.
• Content can be revised and extended easily; now
includes all the maths taught in S154.
• Accompanied by its own set of iCMA questions.
Maths Skills ebook
Maths Skills ebook and iCMA
questions : Evaluation
• Of students replying to an S104 survey, 42.5% had used
the Maths Skills ebook and 97.3% of these had found it
useful.
• The Maths Skills questions were attempted by 1490
people between 28th August 2007 and 24th August
2008.
• Around half of the 118 students who responded to a
feedback question had not looked at the ebook, only the
interactive questions, to ‘tell me what I need to revise’.
Acknowledgements
•
•
•
•
Ruth Williams (S154 Course Team, AYRF)
Linda Fowler (S154 Course Team, AYRF)
Greg Black (LTS, AYRF and Maths Skills questions)
Richard Jordan (Consultant, Maths Skills ebook and
questions)
s.e.jordan@open.ac.uk
Enabling practical work for all
students: using an interactive
screen experiment
Paul Hatherly and Sally Jordan
3rd OpenCETL Conference
The Open University
Practical work is taught by:
• Laboratories at Residential School
• Home Experiments
But there are issues when
students…
• can’t do the experiment for themselves [because of
disability];
• can’t attend residential school;
• need extra practice.
• The role of interactive screen experiments (ISEs):
– Widen accessibility of experimental work;
– Extra practice;
– Improve confidence.
What an ISE is not
Before we can
understand what an
ISE is, we have to
understand what it is
NOT
It is NOT a
programmed simulation
(eg, Hooke’s Law Flash
simulation from PhET,
U of Colorado)
Required physics
is explicitly coded
Much may be left out!
What an ISE is
An ISE is a generalised
very interactive movie
that is delivered remotely
via an ordinary web
browser without special
software
The best way to
understand what
this means is
through an
example…
All physics is
automatically
present
S104 Book 3 Energy and light
Activity 11.1 Investigating light
Students:
• Compare and analyse the spectra produced by an energysaving and a tungsten filament light bulb
Feedback on the experiment
From tutors:
• [students] are normally quite blown away that you can
look inside the atom with just day to day equipment…
• I am really glad that it is compulsory in S104.
From students:
• I really enjoyed this experiment. I was surprised at how
interesting and successful it was. I even got my parents
to observe the difference in the spectrums of the bulbs,
and was able to explain to them how the energy-efficient
bulbs work. It was good because I was able to share my
new knowledge.
• My eyesight is not as good is it used to be.
So we made an ISE…
The S104 ISE
Evaluation of the S104 ISE
We wanted to know:
1. Can students who have used the ISE instead of the
‘real thing’ complete the activity in their course book?
Can they complete related assessment tasks?
2. How easy is the ISE to use?
Students have been observed in the IET Research Lab
and students who use the ISE instead of the real
experiment (and their tutors) will be approached for
feedback.
Early conclusions
• this ISE is easy to use;
• the ‘real thing’ gives a wider experience of practical work
than the ISE, but the ISE is invaluable for those who are
unable to attempt the real experiment.
Note: the ISE is doing something that a simulation couldn’t
Acknowledgements
• John Macdonald (University of Reading)
• Wez Charlton (University of Reading)
Paul Hatherly
p.a.hatherly@open.ac.uk
www.open.ac.uk/picetl/
Supporting student learning
through Tutor Group Forums
Diane Butler
3rd OpenCETL Conference
Tuition and Support
Custom and practice in the science faculty to use
FirstClass forums for support of large groups of students
(course basis)
For S104 we wanted to
• reposition the focus of students onto TGF –tutor group
forums
• make TGFs vehicles for group tuition as well as group
support
Tuition and Assessment
In order to improve students’ perceptions of the value of
online collaboration and deliver an appropriate tuition
strategy for a globally delivered course we
• embedded some online activities into assessments
• allowing us to assess Learning Outcomes relating to
group working and information collation
Tutor Support
To prepare our ALs we produced the following resources
and stored them online on the course web site (Tutor
view)
• A tutor guide to each book of the course outlining both
optional and assessed activities which were suitable for
online tuition and how these were to be delivered
• Detailed teaching resources for student use but guided
by a tutor
The assessed activities
Three assessed activities in S104, all relying on studentstudent collaboration
• Reporting results of a home experiment, sharing and
comparing data
• Accessing specified internet resources for information on
primate species –leading to a discussion on which
species should be targeted for conservation action
(threats, phylogentic importance, geographic range)
• Accessing specified internet resources to investigate the
topic of water on Mars
Optional Activities
• Prepared by AL consultants in conjunction with the
course team
• Wealth of material to use according to identified student
need by the tutor – online or at f2f events
• Material requires facilitation by the tutor –it is teaching
material not student resource
• Aims to sustain activity and tutor-student-student contact
in the TGF
Supporting student learning through
assessment: TMAs and iCMAs
Valda Stevens and Sally Jordan
3rd OpenCETL Conference
Centre for Open Learning of Mathematics, Science
Computing and Technology (COLMSCT)
The Formative Assessment in
Science Teaching (FAST) Project
Gibbs and Simpson identified 11 conditions under
which assessment supports student learning.
(Gibbs, G and Simpson, C (2004) Conditions under which assessment
supports students’ learning. Learning and teaching in higher education, Issue
1, pp3-31)
3 of the conditions relate to pacing and 4 of the
conditions relate to the provision of timely feedback.
Feedback matters!
• Within the Open University context, learners are
geographically separated and we cannot assume that
they will meet their tutor in order to receive feedback.
Feedback on assignments therefore has a particularly
important teaching and learning function.
• We are seeking to provide students with feedback which
is personalised, can be understood by the student and is
received in time to be used in future learning and
assessment tasks.
• So feedback becomes feedforward.
S104 Assessment strategy
• Seven tutor marked assignments (TMAs)
• Eight interactive computer marked assignments (iCMAs)
(summative but low stakes)
• Synoptic component:
Written end of course assignment (ECA)
Longer iCMA (25 questions) with questions ranging across
the course
• We want to help students keep up to date in their
studies;
• Frequent small assessment tasks to help students
monitor their own progress.
The OpenMark system
• Uses a range of question types, going far beyond what
is possible with multiple choice;
• Question types include:
• Numerical input; text input; drag and drop; hotspot
• Students are allowed three attempts with an increasing
amount of teaching guidance, wherever possible tailored
to the student’s previous incorrect answer;
• Different students receive variants of each question so
each has a unique assignment;
• We have included a few questions requiring free text
answers of a phrase or sentence in length.
Evaluation
• We didn’t get the timing of the iCMAs quite right;
• The free text short answer questions are proving to be
remarkably robust;
• There have been some surprises (in student responses
to all questions). It is important to monitor student
responses;
• Students respond to iCMA questions in summative use
in a very different way to when they are used purely
formatively.
Acknowledgments
• The S104 Course Team and our LTS media developers;
• The Centre for Open Learning of Mathematics, Science,
Computing and Technology (COLMSCT) especially
Barbara Brockbank, Phil Butcher and Laura Hills
• Tom Mitchell of Intelligent Assessment Technologies Ltd.
v.a.m.stevens@open.ac.uk
s.e.jordan@open.ac.uk
http://www.open.ac.uk/colmsct/projects/sallyjordan