PhD Research Proposal
Title
Enhancing Student Engagement with Feedback Through Pairwise Comparison:
Investigating Learning Designs that Maximise Educational Impact
Introduction
Feedback is widely recognised as one of the most influential factors affecting student
learning and achievement in higher education. Effective feedback helps students identify
strengths and weaknesses in their work, develop self-regulatory skills, and improve future
performance. Despite its recognised importance, a persistent challenge across university
subjects is that students often fail to engage meaningfully with feedback. Feedback may be
ignored, misunderstood, or viewed as disconnected from future learning activities, limiting its
effectiveness.
Recent developments in educational assessment have highlighted the potential of pairwise
comparison methods as an innovative approach to both assessment and feedback. Pairwise
comparison involves students comparing two pieces of work and judging which better meets
specified criteria. Research suggests that making evaluative judgements about the work of
others can strengthen learners’ understanding of quality, assessment criteria, and their own
performance. By actively involving students in the evaluation process, pairwise comparison
may transform feedback from a passive experience into an engaging learning activity.
This proposal aims to investigate how different learning designs can maximise the feedback
benefits of pairwise comparison activities in higher education. Specifically, it seeks to
identify the conditions under which pairwise comparison promotes engagement, develops
evaluative judgement, and supports academic achievement.
Background and Rationale
The educational value of feedback has been extensively documented. Meta-analyses have
consistently demonstrated that feedback can produce substantial learning gains when
appropriately designed and implemented. However, contemporary feedback research
increasingly argues that feedback should be understood as a process rather than information
transmission. Students must actively engage with feedback for learning to occur.
One promising avenue for increasing engagement is the development of evaluative
judgement, defined as the capability to make informed decisions about the quality of work.
Rather than relying solely on teacher comments, students benefit from opportunities to
evaluate examples of work and compare performance against explicit standards.
Pairwise comparison offers a structured mechanism for developing evaluative judgement.
Unlike traditional marking approaches, pairwise comparison focuses attention on holistic
quality differences between pieces of work. The process requires students to apply
assessment criteria, justify decisions, and reflect on characteristics of high-quality work.
Existing research has demonstrated the reliability and validity of pairwise comparison for
assessment purposes. However, relatively little research has explored how pairwise
comparison can function as a feedback intervention and how learning activities can be
designed to maximise its educational benefits.
Furthermore, there remains limited understanding of the specific design features that
influence student engagement during pairwise comparison tasks. For example:





Does requiring students to justify their comparisons improve learning outcomes?
How does the quality of exemplars affect engagement?
What role do assessment criteria play during comparisons?
How do students perceive pairwise comparison compared with conventional feedback
methods?
Which combinations of learning activities lead to the greatest improvements in
evaluative judgement and performance?
Addressing these questions will contribute both theoretically and practically to feedback
research in higher education.
Research Aim
The overall aim of this study is to investigate how learning designs can maximise the
feedback benefits of pairwise comparison activities in higher education.
Research Questions
1. How do students engage with pairwise comparison activities as a form of feedback?
2. Which learning design features most effectively promote engagement, evaluative
judgement, and learning during pairwise comparison?
3. How do students perceive the usefulness and authenticity of pairwise comparison
compared with traditional feedback approaches?
4. To what extent does participation in pairwise comparison activities influence
subsequent academic performance and self-regulated learning behaviours?
Theoretical Framework
This research will be informed by three complementary theoretical perspectives:
Feedback as Dialogue
Contemporary feedback theory emphasises student participation in feedback processes rather
than passive receipt of information. Pairwise comparison represents a potentially powerful
dialogic feedback mechanism because students actively construct understanding through
evaluation.
Evaluative Judgement
The concept of evaluative judgement provides a framework for understanding how students
learn to recognise quality. Pairwise comparison may support the development of evaluative
judgement by requiring repeated decisions about relative quality and encouraging reflection
on assessment criteria.
Self-Regulated Learning
Self-regulated learning theory suggests that students become more effective learners when
they monitor and evaluate their own performance. Pairwise comparison may enhance selfregulation by helping students internalise standards and identify strategies for improvement.
Together, these frameworks provide a strong conceptual basis for examining both the
processes and outcomes associated with pairwise comparison feedback.
Methodology
A mixed-methods research design will be employed to capture both measurable outcomes
and students’ experiences of pairwise comparison activities.
Phase 1: Exploratory Study
The first phase will investigate student experiences and perceptions of pairwise comparison.
Participants
Undergraduate students from one or more university disciplines will be recruited.
Data Collection




Observation of pairwise comparison activities
Student focus groups
Semi-structured interviews
Reflective student journals
Analysis
Qualitative data will be analysed using thematic analysis to identify factors influencing
engagement, motivation, and learning.
Phase 2: Experimental Investigation
Insights from Phase 1 will inform the design of controlled studies examining different
learning designs.
Potential conditions may include:



Pairwise comparison with written justification versus without justification
Use of annotated versus non-annotated exemplars
Individual versus collaborative comparison activities

Criterion-referenced versus holistic comparison approaches
Data Collection
Measures may include:





Student engagement surveys
Evaluative judgement assessments
Academic performance measures
Learning analytics data
Time-on-task measures
Analysis
Quantitative data will be analysed using appropriate statistical methods such as:




Analysis of variance (ANOVA)
Multilevel modelling
Regression analysis
Effect size estimation
Phase 3: Integration and Design Principles
The final phase will integrate qualitative and quantitative findings to develop evidence-based
principles for designing effective pairwise comparison feedback activities.
These principles will be tested and refined through consultation with students and educators.
Ethical Considerations
Ethical approval will be sought from the relevant university ethics committee.
Key ethical considerations include:





Informed consent from participants
Confidentiality and anonymity of student data
Protection of academic records
Voluntary participation and withdrawal rights
Minimisation of power imbalances between researchers and students
All data will be stored securely and processed in accordance with institutional and GDPR
requirements.
Expected Contributions
Theoretical Contributions
This study will contribute to understanding:



How students engage with feedback through evaluative activities
The mechanisms linking pairwise comparison and evaluative judgement
The relationship between feedback engagement and self-regulated learning
Methodological Contributions
The project will demonstrate how mixed-methods approaches can be used to investigate
feedback processes and educational interventions.
Practical Contributions
The study will provide evidence-based guidance for educators seeking to:




Increase student engagement with feedback
Design effective pairwise comparison activities
Develop students’ evaluative judgement skills
Improve learning outcomes across disciplines
The findings may inform assessment and feedback practices throughout higher education and
support the wider adoption of pairwise comparison as a learning tool.
Timeline
Year 1





Comprehensive literature review
Research training
Development of theoretical framework
Ethics approval
Phase 1 data collection and analysis
Year 2



Design and implementation of experimental studies
Quantitative data collection
Preliminary dissemination of findings through conferences
Year 3





Completion of data collection
Integrated analysis
Development of design principles
Thesis writing
Journal publications and conference presentations
Conclusion
Despite decades of research highlighting the importance of feedback, student engagement
remains a significant challenge in higher education. Pairwise comparison represents a
promising approach that shifts students from passive recipients of feedback to active
evaluators of quality. However, little is currently known about how such activities should be
designed to maximise their educational impact.
By investigating the learning designs that enhance engagement, evaluative judgement, and
achievement, this research will provide both theoretical insights and practical
recommendations for improving feedback practices across university disciplines. The project
aligns closely with current developments in assessment and feedback research and offers the
potential to make a meaningful contribution to educational theory and practice.