Outcomes-based Teaching and Learning
Pre-workshop 3
Designing Assessment Tasks and Grading Criteria
Intended Outcomes of Pre-workshop Exercises
for members of each group
1. Review existing assessment tasks (ATs) in relation to
alignment with subject ILOs.
2. Identify areas requiring changes.
3. Revise or re-design ATs to align to subject ILOs.
4. Discuss and share your ideas with your group
members.
Before the Workshop
Refer to the Pre-workshop reference provided and the revised
subject ILOs in Pre-workshop Exercise 1, conduct Preworkshop Exercises 4, 5 & 6 to:
a. design assessment tasks to address the revised
subject IlOs.
Consider if the ATs require the students to engege in
the ILO verbs? Do the weightings reflect the relative
importance of the ILOs?
b. design grading criteria for either the ILOs or the
assessment tasks.
Discuss with your group members when doing the above
exercises.
Complete the exercises to be ready for discussion and sharing in
the workshops on 2 December 2006.
Outcomes-based Teaching and Learning (OBTL)
using Constructive Alignment
Teaching:
Engaging the
student in the
verb in the ILO
ILO:
What the
student
has to learn
Assessment:
How well
the student
has met the
ILO
Designing Assessment Tasks (ATs)
Steps:
1. Selecting a practicable task that embodies the target
ILO verb.
2. Making a judgment on how well the ILO has been
met by the students' performance on the ATs –
developing grading criteria.
Assessment Tasks (ATs)
.
Provide students the opportunity to demonstrate whether or
not they have achieved the ILOs and what level their
performance is in those ILOs.
. Provide the evidence allowing teachers to make a judgment
about the level of a student’s performance against the ILOs
and to award a final grade.
.
Should be appropriately designed or selected to address the
ILOs that we want to assess.
.
Different assessment tasks address different ILOs.
There would therefore be different kinds of assessment task.
Selection of Assessment Tasks
A range of different assessment tasks may be required to
address the range of ILOs of a subject.
Questions to be asked in selecting assessment tasks:
1. Are the assessment tasks able to address all ILOs?
2. Are the assessment tasks aligned to their appropriate
ILOs? Are the students required to engage in the verbs
identified in the ILOs?
3. Are the assessment tasks practicable with respect to
available time and resources?
4. Do the assessment tasks reflect the relative importance of
the subject ILOs?
5. Is the assessment workload realistic for teachers and
students?
Constructive Alignment of
ILOs and Assessment Tasks
Assessment Tasks
AT 1
AT 2
AT 3
Subject ILOs
Subject ILO 1
Subject ILO 2
Subject ILO 3
Subject ILO 4
Subject ILO 5
Consider if : all ILOs are being addressed?
there is a balanced coverage of the ILOs?
the more important ILOs are given appropriate assessment emphasis.
Common ILOs
Possible Assessment Tasks
Describe
Explain
assignment, essay question exam
assignment, essay question exam,
oral, letter-to-a-friend
project, assignment
case study, assignment
project, case study, experiment
case study, project, experiment
project, experiment
reflective diary, portfolio,
self-assessment
a range of oral, writing or
listening tasks addressing the
ILOs, e.g. presentation, debate,
role play, reporting, assignment,
precis, paraphasing, answering
questions etc.
Integrate
Analyse
Apply
Solve problem
Design, create
Reflect
Communicate
Assessment tasks for the sample subject ILOs
1.
Explain different communicative strategies.
AT: Write an assignment or answer exam questions explaining the four
most important strategies used in report writing.
2. Apply appropriate strategies in different modes of communication.
AT: Write a report on a tutorial case study discussion on a selected topic.
3. Reflect and improve own communicative strategies.
AT: Self-assessment of the report, identifying strengths and areas needing
improvement. Revise the report based on self-assessment.
The three ILOs could be assessed through one assessment task – writing a report
on a tutorial case study discussion followed by a self-assessment and revised
report.
a.
Write a report on a tutorial group discussion;
b.
Explain the strategies used in writing the report;
c.
Self-assess the report, identifying strengths and strategies to be improved.
d.
Revise the report based on c. and submit the revised report together with
the first version.
Examination is a very commonly used assessment task
especially for large classes. We need to consider if
1. examinations involving answering essay type of questions
under invigilated conditions is able to assess students'
performance in some high level ILOs, e.g. apply, reflect, create
etc.
2. there are other alternative assessments tasks which will more
appropriately addess those high level ILOs.
If we must have an invigilated “final exam”, there are far better
formats than the conventional written essays; for example
gobbets, critical incidents (e.g. comment on a video segement),
letter-to-a-friend, and so on.
If we must have exams, consider if the weighting (usually 50% or
more) is appropriate to reflect the relative importance of the
ILOs being addressed.
Exercise 4 - Designing Assessment Tasks
Consider the intended learning outcomes of your subject in
Exercise 1, suggest appropriate assessment task(s) to address
these ILOs. For each of the tasks, explain what the students are
expected to do in completing the task. Are the students required
to enact the target ILO verbs? Discuss with your group
members.
Subject ILOs
Assessment Tasks Student activities in
completing the task
After deciding on the assessment tasks, we have to
consider the following issues.
1. Specification of the tasks
Especially when continuous assessment is used, the different
assessment tasks making up the continuous assessment will
need to specified.
2. Assessment weightings
If assessment tasks are given different assessment weightings,
ensure that these weightings relfect the relative importance of
the ILOs. Take the four sample subject ILOs – explain, apply
reflect and improve. Are they of equal importance? If not,
then the more important ILOs should be given heavier
assessment weighting.
Exercise 5 – Assessment Weighting
Go back to the ATs designed in Exercise 4, now consider assessment
weightings to ensure that they reflect the relative importance of the
ILOs. Discuss with your group members.
Subject ILOs
Assessment Tasks
Student activities in
Wt
completing the task
Does the weighting of the ATs reflect the relative importance of the
ILOs they address?
Grading Criteria
Student performance in assessment tasks should
be judged against a set of clear grading criteria
defining the quality of performance expected of
each of the grades.
Grading ILOs
Direct
(Grading the ILOs)
Grading criteria (rubrics)
of individual ILOs
Derive final grade
Indirect
(Grading the assessment tasks
which are aligned with the ILOs)
Grading criteria (rubrics)
of individual assessment tasks
Derive final grade
Example of Direct Grading Criteria of ILOs
Grade
point/unit
Marginal Pass
D
1.00
Satisfactory
CC C+
1.70 2.00 2.30
Good
B- B B+
2.70 3.00 3.30
Excellent
A- A
3.70 4.00
ILOs
Able to identify a number
Able to identify a full As in “Good” but
write about limited points.
relevant points with some
range of relevant
provides views on
Very little evidence of details. Uses these points
Points with details.
possible alternative
using these points to
to provide a fair reasoning
Supported by relevant causes and/or results
provide reasoning to
or causality. No evidence
literature. Points are
under changing
why they are interof a comprehensive
organized to provide a conditions. Able to
related.
overview of reasoning
comprehensive and
link current
or causality.
cohesive reasoning or reasoning to
causality.
situations in reallife professional
contexts.
Explain Able to identify and briefly
Able to use available
Able to use available As in “Good”. Able
information to selfinformation to selfinformation to selfto generalize selfevaluate and identify
evaluate and identify
evaluate and identify evaluation to beyond
limited aspects of own
more aspects of own
the full range of owm existing context.
strengths and weaknesses
strengths and weaknesses
strengthes and weak- Suggest ways of
in a general sense. No
in a general sense. Little
nesses. Self-evaluation improving performevidence of suggestions
application of theory in
is based on theory.
ance to real-life
of ways to improve
self-evaluation and limited Increasingly able to
professional
performance. No evidence
suggestions of ways to
suggest ways to
contest.
of theory being used in
improve performance.
improve performance
self-evaluation.
in a specific context.
Reflect Able to use available
Example of Indirect Grading Criteria of Assessment Tasks
in a Portfolio
Marginal Pass
D
1.00
Satisfactory
C- C C+
1.70 2.00 2.30
Good
B- B
B+
2.70 3.00 3.30
Excellent
A- A
3.70 4.00
The pieces of evidence The evidence is relevant,
The evidence presents a As in “B” but with
are relevant and
accurate and covers
good appreciation of
higher degree of
accurate, but are
several aspects of the
the general thrust of the originality and
isolated, addressing
subject. Little evidence of subject. Good coverage
evidence of interone aspect of the
an overall view of the
with relevant and
nalization into
subject. Demonstration subject. Demonstrates
accurate support. A clear personalized model
of understanding in a
declarative understanding view of how various
of practice. Good
minimally acceptable
of a reasonable amount of aspects of the subject
evidence of reflectway. Poor coverage, no content. Able to discuss
integrate to form a
ion on own
originality, weak
content meaningfully.
thrust or purpose.
performance based
justification of portfolio Good coverage but little
Good evidence of
on theory.
items. Inappropriate
Application or integration. application of subject
Generalizes subject
self-evaluation.
Fair justification of items. Content to practice.
content to new and
Attempted realistic selfPortfolio items well
unfamiliar realevaluation
justified. Realistic
life contexts.
self-evaluation.
Departments need to decide whether to use
direct or indirect grading of the ILOs.
A holistic way of deriving a final grade
from multiple grades
ILOs
1. Relate
AT1
2. Apply
3. Identify
achieved
AT2
√
AT3
AT4
Grade
A – if all ILOs well
achieved but 1 & 2
excellent
√
√
√
√
B – if all are well
4. Explain
√
5. Communicate
√
C – if 3, 4 & 5 are well
achieved
D – if marginal or only
2 ILOs are achieved
An example of a holistic way of deriving final grade
Curriculum and Instruction: A subject in a course for Ed. Psychlsts.
Grading will be based on your attaining the following ILOs
1.
2.
3.
4.
Apply the principles of good teaching and assessment to chosen contexts.
Relate selected aspects of curriculum design and management to the educational
system in Hong Kong.
Apply the content and experiences in this subject to enhance your effectiveness as
an educational psychologist.
Show examples of your reflective decision-making as an educational psychologist.
Final grades will depend on how well you can demonstrate that you have met all the
ILOs:
A
B
C
F
Awarded if you have clearly met all the ILOs, provide evidence of original and
creative thinking, perhaps going beyond established practice.
Awarded when all ILOs have been met very well and effectively.
Awarded when the ILOs have been addressed satisfactorily, or where the evidence
is strong in some ILOs, weaker but acceptable in others.
Less than C, work plagiarised, not submitted.
Example of Holistic Grading of Assessment Tasks
in a Portfolio
Marginal
D
1.00
Adequate
C- C C+
1.70 2.00 2.30
Good
B- B B+
2.70 3.00 3.30
Excellent
A- A
3.70 4.00
The pieces of evidence
The evidence is relevant, The evidence presents a As in “B” but with
are relevant and
accurate and covers
good appreciation of
higher degree of
accurate, but are
several aspects of the
the general thrust of the originality and
isolated, addressing
course. Little evidence of course. Good coverage
evidence of interone aspect of the
an overall view of the
with relevant and
nalization into
course. Demonstration
course. Demonstrates
accurate support. A clear personalized model
of understanding in a
declarative understanding view of how various
of practice. Good
minimally acceptable
of a reasonable amount of aspects of the course
evidence of reflectway. Poor coverage, no content. Able to discuss
integrate to form a
ion on own
originality, weak
content meaningfully.
Thrust or purpose.
Performance based
justification of portfolio Good coverage but little
Good evidence of
on theory.
items.
Application or integration. application of course
Generalizes course
Fair justification of items. Content to practice.
content to new and
Portfolio items well
Unfamiliar realjustified.
life contexts.
Exercise 6 – Developing Grading Criteria
Consider the intended learning outcomes and assessment
tasks you have identified in Exercise 4, suggest
appropriate grading criteria addressing either (a) the
ILOs or (b) the assessment tasks. Discuss with your group
members.
Subject ILO and AT:
Grade
Grading criteria
Help our students understand the grading criteria
. Remind students of the ILOs to be assessed.
. Explain to students the standards expected of each grade.
. Illustrate with previous assessment tasks on
How the grading criteria have been applied.
How grades have been awarded.
How a better grade could have been achieved.
. Clarify any problems that students may have.
After you have completed the pre-workshop
exercises,
1. select one group member to share your
ideas at the workshop, and
2. jot down and issues that you would like to
bring up for discussion at the workshop.