Mechanised Assessments that
Promote Deep Learning
…and prevent plagiarism!!
Gareth Denyer & Dale Hancock,
University of Sydney
Multiple Choice
• Rapid, convenient
– Excellent for large classes
• BUT…. do they encourage surface learning?
– Questions often designed to test unambiguous facts
• Student PERCEPTION
– adopt a ‘rote’ approach
– Names, facts, numbers
• Attempts at deeper, conceptual based possible…
– BUT… often lead to ambiguous questions
– At worst, questions favour less able students because better
students confused!
• VERY SKILFUL JOB!!!
Marking MCQs
• No credit for near misses or process
– No point in leaving answers ‘blank’
• So you never know ‘why’
• Guessing can give 20%
– Negative marking?
• Intimidates students
• Forces meticulous non-ambiguity
• Further driving surface learning!
• Post-Exam revision of mark scheme difficult
– Unlike SAQs
• where mark scheme can be dynamically changed
– Questions often ‘discarded’
• Disadvantages good students
Graded Alternative
NB. This is a SURFACE example!!
What is the capital of Australia?
A
B
C
D
E
Melbourne
Sydney
Glasgow
Canberra
Auckland
0.5
0.5
-1
1
0
– The more complex the question, the more
important partial marks become.
XL Based Graded Solution
Typical
student
answer grid
Marks
Key Sheet
Calculations
•
Blah, blah… What is the approximate
ε450nm for the coloured compound formed
in the assay?
A.
B.
C.
D.
E.
0.199 mM-1 cm-1
250 mM-1 cm-1
4 mM-1 cm-1
2 mM-1 cm-1
19.6 mM-1 cm-1
Each option reveals a
particular mistake
Especially good with a
multi-step calculation
Can reward process
Opens up Deeper Styles
• The “What if?” Cause → Effect
– What would be the consequences of inhibition
of lipolysis during the first few days of
starvation?
• Shades of Grey
– You decide that you need 50 mU of citrate
synthase (CS) in the cuvette. What is the
MOST PRACTICAL way of doing this?
Advantages
• Makes marking flexible
– Like an SAQ!
– Post-hoc changes possible
– Provides an alternative way of scaling
• Staff less stressed
– More extrapolative questions
• Less attention to rigorous ‘fact’
• Less concern about ‘getting it perfect’
– No conflict with colleagues over ambiguous questions
• Feedback better
– Each option “useful”
Plagiarism!
• Much attention given to detecting cheating
in assignments
– Puts you into a conflict situation
– But exams make up most of the marks!
• Cheating is easy in MCQ exams
– Hand signals
– Pattern recognition
• Several cases
– A healthy literature!
Islands of
corruption
20
15
Similarity of
Incorrect Answers
10
5
10
7
0
4
1
2
3
4
Column
5
6
7
Row
1
8
9
10
adapted from data in: Harpp & Hogan (1993) Crime in
the classroom: Detection and prevention of cheating on
multiple-choice exams. J. Chem. Educ. 70, 306
Wise Words
Universities obviously want to
stamp out cheating
But they also do not want any
students to be falsely accused
1 in 653,000
Statistical evidence is
generally not enough.
Pursuing a case is traumatic
for all those involved
Especially if an unwilling cheatee!
The Versions Solution
Create FOUR versions of
the same paper
Questions in same order,
options just rotated
Layout so that each
version is isolated from its
clones
Very easy in Word
A database solution can
generate even more
versions.
Implementation
• Sweet Justice!!
– Appropriate recompense without confrontation
– Lots of ‘evidence’
• Sometimes even ‘check’ questions copied
• Easy
– Just Word and Excel
– Rotations take about 10 min per 100 MCQs
Take Home Messages
• Graded MCQs give flexibility
• Different versions disincentivise cheating
• All easily implemented
– Word and Excel
• We are happy to provide templates and
instruction
Have a go! Easy and Liberating!