Formative Quizzes and their
Contribution to the Understanding of
Computer Programming by Level
One Students
Michael Isherwood
Associate Teaching Fellow, COLMSCT
Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT)
Centre for Excellence in Teaching and Learning
The Open University is incorporated by Royal Charter (RC 000391), an exempt charity in England & Wales and a charity registered in Scotland (SC 038302).
Presentation Summary
• Project Background
• Indicators of Success?
• For the Future
Background
• Students find getting to grips with programming difficult
• This leads to frustration and dropping out of M150 (and
therefore further computing courses)
Background - TMA02/03 Marks
A Solution - iCMAs
• A series of Interactive Computer Marked Assessments
• Optional but Accessible through M150 Website
• But
– Also other initiatives – podcasts and video files
– Ian Cooke’s Project
– Course Team Initiatives on Assessing Programming
Dropout and TMA03 Skipping
No of Students
engaging
% drop out
after TMA01
% drop out
after TMA02
% skipping
TMA03
2008B
1262
9.1
13.4
7.8
2008J
2213
9.8
9.9
2.7
2009B
1486
8.5
9.6
3.8
All Three
4961
9.2 (7.6)
10.7 (10.5)
4.3 (5.3)
Bracketed figures for pre iCMA presentations: 2006B – 2007B
• 2008J and 2009B has iCMAs accessible from M150 Website
• Fewer skipping TMA03 than previously (2008B aberrant)
• Marginally fewer dropping out after TMA02
• Final figures for completions of 2008J and 2009B not available as
resits pending
Logical AND Question
CP2B A steel bar of nominal length 220mm needs to be less than
1mm different from this length. To be within tolerance, its length must
therefore be between 219mm and 221mm. Write a compound
condition that evaluates to true for being within tolerance, using len
for length (in mm).
Free text answer is: (len > 219) AND (len < 221)
• Student needs to know form of expression and have the correct
Boolean operator (AND) as well as the correct conditions
(bracketed parts)
• OpenMark identifies each of these components, recognises errors
and gives targeted feedback on these before the student has a
second attempt
Learning on Similar Questions
• A series of questions has been set using the logical operators
AND and OR
• In Quiz 2 there were two similar questions. The logical operator
was AND in the first case and OR in the second case. (The Qs
were of the same format)
• In Quiz 3 there was a similar question requiring selection of either
AND or OR
• There was more than described here to each of the questions
/%
AND
OR
Other
1
62
19
19
2
28
70
2
3
25
74
1
• First question has many still learning the requirement
• By third question still improvement
Units in Conditional Statements
• The two Quiz 2 Logical Operator questions incorporated
units in the wording of the question.
• The units depending on variant were g mm nm and oC
• In the first question 10% of students incorporated the
unit and in the second, it was 12% - no improvement!
Learning through Attempts
• Students have 3 attempts per question
• After each attempt, targeted feedback is given based on
the student’s answer
%
1st Try
2nd Try
3rd Try
Still Wrong
32
15
8
45
OR question used earlier
• It is evident when looking at an individual student’s attempts that
most learn on their 2nd and 3rd attempts
• However this is often not true for those who are wrong after the
3rd attempt
Weak Students – 3 Wrong
Attempts
• 37% with wrong Boolean operator – of these 75%
correct on 2nd attempt BUT 25% don’t despite pretty
explicit feedback
• 13% have all 3 attempts identical and a further 38%
have two attempts the same
• 10% had little or no idea with 3 “garbled” attempts and
2% left completed blank
The Student View
• Feedback from Students indicates that most find the
quizzes both enjoyable and useful
• Many Students also wanted more
• There is more on this in Poster Presentation
For the Future
• Need many more questions available for students who
need to develop their skills
• Needs routes through quizzes that are determined by a
combination of that student’s performance and that
student’s choice – differentiated learning
• Tutors need to be able to monitor progress and, if
necessary, intervene
• Much more time needed for analyses of student
performance and needs
Thank you for Listening