Quality Improvement of First Year
Assessment:
A process and template for the review &
enhancement of assessment design &
management
Margaret Macleod
Keithia Wilson
Griffith University
Acknowledgment to Country
In the Spirit of Reconciliation
Following on from Sorry Day
I would like to acknowledge & honour the
Traditional Custodians of the land we are
meeting on today, Turrbal and the Jagera
People, and pay respect to their Elders past &
present
And acknowledge the contribution of our First
Nation Peoples to Higher Education & learning
Keithia Wilson GU - May 2012
Introductions
Margaret Macleod:
• Currently the Griffith Health Group
Assessment Consultant
• 10 years previous experience as Learning
Adviser at Griffith University, Logan campus
Prof Kethia Wilson:
• Program Director of the FYE, Griffith Health
• ALTC National Fellow for the FYE (2010-2012)
Session Overview
1. Mini-input





Context for the intervention
Literature - Importance of assessment
Assessment review process
Assessment review template
Evaluation & Outcomes
2. Reflection & group discussion
3. Final summary of themes
Griffith University context
• Large metropolitan university in Brisbane (1 of 7 in S-E
Qld, & 1 of 4 in Brisbane)
• Multi-campus - 5 campuses x 60 k corridor
• Student enrolment of 40,000
• 70% of students are first-in-family at uni - FIF correlates
with low SES & lower entry scores
• 6th highest low SES student intake in Australia
• 3rd highest Indigenous student intake in Australia
• 25% International student enrolment
• 10% students studying in distance mode
Keithia Wilson GU - May 2012
Why assessment matters!
There is very strong evidence in the literature to suggest –
• Assessment is a central feature of curriculum
• Assessment practices have a profound effect on
student behavior (Entwistle, 1991)
• Assessment is believed to have a more powerful
influence on student behavior than teaching (Boud,
2012)
• Assessment has also been identified by leading
educators as one of the least sophisticated aspects of
learning and teaching in higher education (James,
2010)
In Summary
“While students can with difficulty
escape from the effects of poor
teaching, they cannot (by definition
if they want to graduate) escape the
effects of poor assessment”.
(David Boud, 1995, p 35)
Effects of Poor Assessment
The effects of poor assessment have been well
documented in the literature (Rust, 2002) –
• Student disengagement from learning, and
• The development of negative attitudes
towards learning
Impact of poor assessment on NonTraditional students
Anecdotal & observational data from Professional
staff (e.g., Learning Advisors) indicates that poor
assessment contributes to students, particularly
non- traditional students, experience
• a sense of personal inadequacy,
• disempowerment,
• perceived inability to negotiate the complexities
of university culture, and
• lower retention rates.
First Year Students’ Appraisals of
Assessment
Wilson & Lizzio (2012) ALTC Grant
Commencing students evaluate their
assessment tasks in terms of two factors:
1. The motivational value of the task, and
2. Its manageability, or the extent to which the
task is scaffolded, with resources provided to
enhance engagement and understanding.
Key Findings for FY Assessment
• Students’ perceptions of the motivational value
of assessment tasks are key in predicting student
engagement,
• Students’ perceptions of the effectiveness of the
design of the assessment task are key in
predicting their levels of motivation for the task,
• Student evaluations of task manageability are key
in predicting student’s sense of self-efficacy or
confidence in undertaking the assessment task.
Dual Assessment Factors guiding the
Review Process
Thus, the dual factors of
1. Effective /appropriate task design
2. Combined with processes for managing the
assessment experience and scaffolding student
understanding of the assessment task are core
to a positive, empowering and potentially
successful student experience with assessment
Reinforced by feedback from Professional staff, in
particular, Learning Advisors
Common problems students
experience with assessment
• Difficult, ambiguous or complex tasks
• Tasks that lack relevance to learning outcomes or
‘sense of purpose’
• Misleading or generic marking criteria
• Inconsistencies in task information
• Insufficient resources to help scaffold tasks
• Inappropriately timed tasks
Review Process Overview
Academic-professional staff partnership
– Worked together to design and implement a process to review and
enhance first year assessment, and work effectively with academic
staff
– Macro (FY Program) and micro (Individual tasks) level review and
enhancement of first year assessment
– Undertaken in two Schools in Griffith Health in 2010
Strategy:
 Focus on student voice and understanding of student experience of
assessment
 Academic staff engagement
 Design of template to evaluate written assessment tasks
 Report, including recommendations and resource development
 Change process combining FYE Team & Individual Convenor
discussions for decision-making at FY Program & Course/Unit levels
The Review Process: Design of
Templates
• Macro level Template
– Programmatic focus:
• comparison of all assessment tasks re timing, type, variety,
word length, weighting
• Micro Level Template
– Individual assessment task focus:
• Assessment design
• Scaffolding of tasks and targeted Resources
Final report: Overall summary and detailed
evaluation of written assessment in all first
semester, first year courses in each School
Assessment Design
1. Relationship to Course Learning Outcomes: Does the assignment task
clearly relate to/develop the learning outcomes for the course?
2. Clarity: Is assignment task clear and unambiguous?
3. Criteria: Do marking criteria assist in understanding the assignment task?
4. Level of difficulty: Is assignment task suitable for students’ level of skill
development?
5. Weighting: Is weighting appropriate for assignment task?
6. Size of task: Is word length appropriate for assignment task?
Assessment Resources
Are there sufficient resources provided to assist students with this task?
Suggestions for Scaffolding & Enhancement of student learning
Assessment redesign
Resource enhancement
Margaret Macleod, GU, 2011
The Review Process: Academic staff
engagement
1. Ensuring academic support:
Creation of FY leadership team, comprising Dean L&T (Health), HOS/DHOS,
Degree Program Convenor/s, First Year Advisors, & FY Course/ Unit
Convenors
Creation of FY Enhancement/Management team, comprising FY Leadership Team
+ FY Course/Unit Convenors
2. Facilitating staff buy-in & engagement:
Consultation process with FY staff prior to review to seek commitment &
agreement (or otherwise) to the audit process
3. Conducting the Assessment Review:
Accessing Course/unit outlines for analysis
Preparation of Final Report
4. Facilitating the feedback process:
Macro level: programmatic discussions with FY leadership teams to discuss report, share
information & make FY Program decisions
Micro level: Learning Adviser met with individual course convenors to assist in redesign of
assessment & creation of resources to scaffold student engagement & learning
Assessment Review Outcomes:
Macro level audits
Program Level changes
• Relocation of some more difficult courses & tasks to 2nd year (e.g.,
research methods)
• Reduction of high stakes, highly weighted (e.g., 40 to 50%) first
assessment tasks in favour of early, formative low stakes assessment tasks
in most courses
• Agreement in FYE Teams on total word lengths for semester 1 tasks (from
5-6,000 to 4,000 words);
• Reduced standardised weightings for word length (e.g., 1,500 word range
from 15 to 45%);
• Reduction in number of group work tasks for assessment purposes & later
submission dates (not before week 6)
• Reduction in number of referencing styles (from 2/3 to 1)
• Coordination of submission dates
• Blackboard Audit resulting in consistent information storage
Assessment Review Outcomes:
Micro level audits
Individual assessment task changes:
• Total redesign of some assessment tasks
• Partial redesign of many tasks re task clarity, level of
difficulty, variety, relation to course aims, marking
criteria
• Development of task specific resources to scaffold
student learning
Assessment Review Evaluation
Staff and student outcomes: evaluated using data from
commencing students (Starting@Griffith Survey – week
7/semester 1) and feedback from teaching staff (end sem 1)
• Student feedback: Increases in 2011 from 2010 in
–
–
–
–
–
student engagement between 5 & 7%
student satisfaction between 6 & 8%
student capability up by 10%
perceptions of good teaching between 11 & 14%
These were the highest in the university
• Academic staff feedback:
– Substantial reductions in student enquiries, student anxiety, &
failure rates in threshold courses/units
– Positive perceptions of process – requests that review process
be continued into 2nd and 3rd years.