Is our assessment up to standard?
David Boud
University of Technology, Sydney
and
Centre for Research in Assessment and Digital Learning,
Deakin University
University of Sydney Teaching Colloquium, 29-30 September 2014
Overview
Part 1.
What are academic standards now and why
should we bother with them?
Part 2.
The changing ground of assessment
Part 3.
What are the implications of the
standards agenda for assessment?
Part 4.
What else is there to consider?
Part 1
What are academic
standards now and why
should we bother with
them?
What do we now mean by academic
standards?
From
positional rhetoric:
‘we are committed to maintaining high
academic standards’
to
something we are judged by:
‘are your assessment practices up to standard?’
From where do academic standards
arise?
• Intrinsic to the structure of
disciplinary knowledge
• consensus views of experts
• analysis of actual
professional practices
From whom do they come?
•
•
•
•
Ourselves/our discipline
Professional bodies
Accreditation agencies (eg. AACSB)
International agreements (eg. AHELO)
The new landscape of academic
standards
•
•
•
•
•
•
•
AQF
TEQSA
The OECD move
Professional registration bodies
Accrediting agencies, eg. AACSB
Threshold discipline learning outcomes
Most importantly, ourselves, but beyond
the implicit standards of disciplines
• Underpinning this is empirical justification
of standards and comparability
NB: most TEQSA standards are not about
academic matters at all!
Part 2
The changing ground of
assessment
What does assessment need to do?
• Contribute to certifying
student performance
– Summative assessment
• Provide students with useful
information to aid their
learning now
– Formative assessment
• Build students’ capacity to
make judgements about
their own learning
– Sustainable assessment
Managing the complexity of demands
• One activity cannot meet all the requirements of
assessment
– This is not a simple matter of having a diversity of
methods
• Some purposes are incompatible because of
timing or type of information needed
– Eg. grades versus rich information
• In any given instance one purpose needs to be
dominant
– This needs to change over the progress of a course or
program
Assessment decision-making framework
assessmentdecisions.org/ decision framework
Which of the following will you take into account in
your design for assessment?
– Learner characteristics
– Institutional assessment policies
/conventions
– Professional, vocational
requirements
– Departmental, disciplinary and
personal ‘signatures’
– The overall program
– Learning environment
Part 3
What are the implications of the standards
agenda for assessment?
The context of assessment today
• Assessment carries the burden of many sets of
expectations
• Academics mediate these to ensure courses
are worthwhile and are not captured by any
one interest
• Assessment decision-making, is about
balancing different sets of considerations,
with a core of ensuring students have high
quality experiences
What is assessment
(in the context of learning outcomes)?
• Assessment is a judgement about whether students can
demonstrate attainment of learning outcomes to a given
standard.
• Transparent standards must be established for assessment
tasks
– Setting a pass mark is not setting a standard
– Setting a general set of standards for a course is not enough
– Use of terms such as good, superior, excellent does not indicate
a standard or communicate a level
• All assessment must be standards-based (criterionreferenced)
– Norm-referencing is excluded
– Judging students against each other undermines standards
What does this imply?
• All assessment involves the
identifying appropriate standards
for the tasks students undertake
• A range of assessment
approaches must be used
appropriate to the range of
learning outcomes to be assessed
• The balance of assessment
approaches must reflect the
range of learning outcomes
• Ensuring all the necessary
outcomes are met by all students
What does it not imply?
• Standards are unilaterally
applied
• Students are not involved
in assessment
• All learning can be
predetermined
• All learning can be easily
measured/judged or is
worthwhile
• Existing use of marks and
grades is compatible with a
standards framework
Part 4
What else is important?
Standards implies fixity, but
how can we be responsive?
The role of feedback: what can it do?
• Bridges the gap between teaching and
learning, ensuring the curriculum is adjusted
to the needs and learning of each student
• Cannot be enacted without the engagement
of participants—students and teachers.
• Needs to respond to what students actually
do.
What is feedback?
“a process whereby learners
obtain information about their
work in order to appreciate the
similarities and differences
between appropriate standards
for any given work, and the
qualities of the work itself, in
order to generate improved
work”
Boud and Molloy 2013
Properly understanding feedback
Feedback is about effects not inputs
• Feedback is not synonymous with the comments
we provide to students’ about their work
• Feedback cannot be said to have occurred unless
a student’s work is positively influenced
Implies: course units need to be designed to allow
for multiple feedback loops to be incorporated
within a semester.
Feedback as Iterative Task Design
Nesting of tasks enables feedback loop to be completed through knowledge
of the effects of earlier information provision in subsequent tasks. From Boud
and Molloy (2013)
Degree of
task
challenge
Overlap of
learning
outcomes
Overlap of
learning
outcomes
Activity 3
Activity 2
Activity 1
Time through semester
What do/can students do?
• Be expected to act on feedback
– at least to respond to it
• Ask for particular feedback
– to position them as active players
• Engage in a following task in areas in which they
have not reached an appropriate standard
• Be expected to monitor their own achievement
The feedback-enabled curriculum
• Has early strategies to shift learner identity to becoming selfregulated
• Positions feedback as part of learning, not as an adjunct of
assessment
• Equips students to be skilled and comfortable with
negotiating learning outcomes, feedback processes and
information needs
• Fosters ongoing ‘dialogue’ between students and teachers
about feedback processes, the nature of standards and the
practicing of judgement.
• Introduces activities to enable students to calibrate
judgement (of their own work and that of others)
Finally,
being responsive is not
just about effective
feedback
Student engagement in assessment
Why involve them?
• They need to know how to judge the work of
themselves and others
• They need to appreciate standards and criteria
for good work to be effective learners
• They can’t act effectively unless they know
what they know/don’t know
Every act of assessment should build
students’ capacity to judge work
• The most basic outcome of any course is that
students can judge good work
• This capacity needs to be developed over time
in conjunction with the assessment of tasks
Implies: Design the activities involved in each
task to enable students to develop skills of
judging their own work and that of others
How can students be more actively
involved in assessment?
Applicable to all students
• Through not being the passive recipients of assessment
acts
• Through choosing assessment tasks appropriate to
learning outcomes
• Through identifying standards and developing suitable
criteria by which to judge their work
• Through acting as sources of feedback information for
others
• Through being expected to act on feedback in
subsequent work
Realising the potential of assessment
• Assessment continues to be the single most
powerful influence on learning
• Attention to assessment tasks and what students
(and we) do in association with them is more
likely to shape students positively than anything
else we can do
• Assessment standards are so important, we can’t
maintain a monopoly on ensuring they are
applied: responsibility for ensuring standards are
met is a continuous responsibility for all
practitioners, including learners
Some criteria for good assessment
1. Together, assessment activities address the different
purposes of assessment
2. Linked to full range of unit and (agreed) program
outcomes
3. Enables multiple feedback loops to be utilised
4. Aligned with learning activities, not just presentations
5. Tasks are worthwhile/significant in their own right (not
unnaturally contrived for assessment purposes)
6. Enables judgements to be made about whether all
students have met given standards
7. Equips students to judge their own work
8. Builds students’ capacity to continue to learn
Resources
• Assessment decision-making for course units
OLT project: assessmentdecisions.org
• Feedback
University of Edinburgh
tla.ed.ac.uk/feedback/index.html
Re-Engineering Assessment Practices in Scottish Higher Education
reap.ac.uk
• Assessment for future learning
OLT Fellowship: assessmentfutures.com
Remove spurious levels of accuracy
• Use of percentages implies detection of
variations impossible to make
• Never grade on a scale more fine-grained than
the distinctions that can reasonably be made
• Summative assessment is about judging if
learning outcomes have been met, not making
measurements of detailed performance on a task
Implies: are more than the four common passing
grades ever needed?