Computer-marked assessment or
learning analytics?
Sally Jordan
Department of Physical Sciences
The Open University
VICE/PHEC 29th August 2014
Background
• Definitions of learning analytics e.g. Clow (2013, p. 683):
“The analysis and representation of data about learners
in order to improve learning”;
• But assessment is sometimes ignored when learning
analytics are discussed. Ellis (2013) points out that
assessment is ubiquitous in higher education whilst
student interactions in other online environments are not;
I will also argue that analysing assessment behaviour
also enables us to monitor behaviour at depth;
• Assessment literature is also relevant e.g. Nicol &
Macfarlane-Dick (2006) state that good feedback
practice “Provides information to teachers that can be
used to shape teaching”.
Analysis of student errors
• Can look at the individual student or the cohort level;
• At the individual level, this can form the basis of
diagnostic testing;
• At the cohort level, look for questions that students
struggle with;
• Look at responses in more detail to learn more about the
errors that students make;
• This can give insight into student misunderstandings.
So what topics in Maths for
Science do students find difficult?
So what topics in Maths for
Science do students find difficult?
Analysis of student responses to
individual questions
Gives information about student errors, linked to
their misconceptions. The confidence in the
findings is increased when
• The questions require a ‘free-text’ (constructed)
response;
• The questions are in summative use (students
are trying);
• Similar errors are seen in different variants and
different questions.
See Jordan (2014a)
Analysis of engagement
…“750 students used my quiz”
When do students do quizzes?
(Overall engagement)
When do students do quizzes?
(impact of hard deadlines)
When do students do quizzes?
(typical patterns of use)
Student engagement with
feedback
Student engagement with
feedback (identical question)
Module A
Module B
See Jordan (2014b) and Jordan et al. (in press)
The future?
Redecker, Punie and Ferrari (2012, p. 302)
suggest that we should “transcend the testing
paradigm”; data collected from student interaction
in an online environment offers the possibility to
assess students on their actual interactions rather
than adding assessment separately.
References
Clow, D. (2013). An overview of learning analytics. Teaching in
Higher Education, 18(6), 683-695.
Ellis, C. (2013). Broadening the scope and increasing the
usefulness of learning analytics: The case for assessment
analytics. British Journal of Educational Technology, 44(4), 662664.
Nicol, D. & Macfarlane‐Dick, D. (2006). Formative assessment
and self‐regulated learning: a model and seven principles of good
feedback practice. Studies in Higher Education, 31(2), 199-218.
Redecker, C., Punie, Y., & Ferrari, A. (2012). eAssessment for
21st Century Learning and Skills. In A. Ravenscroft, S.
Lindstaedt, C.D. Kloos & D. Hernandez-Leo (Eds.), 21st Century
Learning for 21st Century Skills (pp. 292-305). Berlin: Springer.
For more about what I’ve discussed
Jordan, S. (2014a). Adult science learners’ mathematical
mistakes: an analysis of student responses to computermarked questions. European Journal of Science and
Mathematics Education, 2(2), 63-87.
Jordan, S. (2014b). Using e-assessment to learn about students and
learning. International Journal of eAssessment, 4(1)
Jordan, S. E., Bolton, J. P. R., Cook, L. J., Datta, S. B., Golding, J.
P., Haresnape, J. M., Jordan, R. S., Murphy, K. P. S. J., New, K. J.,
& Williams, R. T. (in press). Thresholded assessment: Does it
work? Report on an eSTEeM Project. Will be available soon from
http://www.open.ac.uk/about/teaching-andlearning/esteem/projects/themes/innovativeassessment/thresholded-assessment-does-it-work
Sally Jordan
Senior Lecturer and Staff Tutor
Head of Physics
Department of Physical Sciences
The Open University
sally.jordan@open.ac.uk
blog: http://www.open.ac.uk/blogs/SallyJordan/