Example 1: Rubric for a short answer test question

advertisement
MARKING AND GIVING FEEDBACK
Introduction to marking
Using rubrics to mark accurately and reliably
Estimated duration: 60 minutes
Carry out some further research on rubrics and gather some inspiration by looking at the following:
 The examples below showing rubrics of different types.
 Brown, G, Bull, J, & Pendlebury, M. (1997) Assessing student learning in higher education. New York:
Routledge.
 Walvoord, B. & Anderson, V. (1999) Effective Grading Practices. San Francisco, CA: Jossey-Bass.
Examples of rubrics
Contents
Introduction ................................................................................................. 2
Example 1: Rubric for a short answer text question ................................................ 3
Example 2: Rubric for scoring writing (1) ............................................................. 4
Example 3: Rubric for scoring writing (2) ............................................................. 6
Example 4: Rubric for scoring student presentations (1) ........................................... 7
Example 5: Rubric for scoring student presentations (2) ........................................... 9
Example 6: Rubric for scoring an exam question .................................................. 10
Example 7: Rubric for first component of proposal paper ....................................... 13
Example 8: Rubric for quizzes and homework ...................................................... 14
Example 9: Rubric for evaluating senior design project .......................................... 15
Example 10: Rubric for problem-based learning discussion lab ................................. 17
Example 11: Using rubrics to help students develop their own
ability to assess their work ......................................................................... 18
1
Introduction
The examples of rubrics given in this document demonstrate how varied they can be. For example, levels of
performance can be expressed in a range of ways…

They might be laid out on a scale with the points along the scale given descriptive words, like this:
0
1
2
3
Unacceptable
Minimally acceptable
Acceptable
Exceeds expectations

They might be more elaborate and informative, like this:
Unacceptable
The answer has several
errors in both conceptual
level and communication
clarity. Multiple
mechanical errors or
complete failure to
answer the question
Minimally acceptable
The answer contains
one or two basic facts
that are correct, but
may also have incorrect
statements as well. No
connections or
comparisons provided.
Acceptable
The answer
contains most
(75%) of the points
that needed to be
included. The
writing is clear, if
uninspired.
Correct attempts to
integrate the
points.
Exceeds expectations
The answer not only
contains the main
points but goes beyond
them to provide a
critique of their
veracity. The writing is
clear and measured.
Rubrics frequently indicate the point value or level in addition to or in place of the scale shown above, like this:
Component
All main ideas are included
Ideas have been analyzed
Ideas have been connected
Writing is clear and coherent
Possible point value
5 points total
5 points total
3 points
3 points
As you read through the examples, make a note of elements that you consider particularly effective or
relevant to your own situation.
Reflective questions
You may also wish to reflect upon the following questions as you read each rubric:
 How much room does the rubric leave for individual interpretation or subjectivity in marking? Can you see
any ways of making it more specific?
 Imagine using each rubric. Are there any areas you feel would require further clarification before you were
able to make a start? Is it clear how you are to decide between each level of attainment?
 What is an appropriate level of detail for a rubric? Some academics have argued that, while it is helpful to
be as minutely specific as possible, there is also value in taking a step back and marking more holistically,
and that the final grade should reflect a balance between these two approaches. Do you agree? Why
(not)? And where on the spectrum of minutiae/holistic does each rubric stand?
Our thanks to all those who volunteered examples from their own courses.
2
Example 1:
Rubric for a short answer test question
Source: Marilla Svinicki, University of Texas at Austin College of Education
Introduction
This rubric might be used for scoring a written essay exam question. Note that there is space for the grader to
keep track of unanticipated responses and how they were scored so that all papers will be consistent any time
the unanticipated answer is encountered.
Question
Explain in 100 words or less how you would apply the theory of vicarious reinforcement to exercise more
control over students in an elementary level class. (10 points)
Rubric
Criteria
Accurate description of vicarious reinforcement as a
concept
Accurate use for classroom management suggested
Complete, well-structured answer less than 100
words
If incorrect terminology used, subtract
Points
3
5
2
-3 if frequent, -2 if occasional
Notes to grader
Frequent misconceptions used (and disposition of each instance)
Frequently used incorrect terminology (and disposition of each instance)
3
Example 2:
Rubric for scoring writing (1)
Source: Sally Kuhlenschmidt, Western Kentucky University
Introduction
This more detailed rubric is designed for assessing writing assignments.
CATEGORY
Most points
Zero points
Clarity
Grammar, spelling,
and style make it easy
for the reader to
follow. Uses words
correctly and avoids
jargon unless it is the
most precise word.
Occasional (2 or 3 per
page) grammar, spelling
or style problems.
Tendency to use vague
words or excessive
jargon.
Accuracy
All statements are
accurate relative to
the resource being
examined and/or to
the material in the
textbook.
Discrepancies
between sources are
indicated.
All statements are
No more than one minor
accurate relative to the inaccuracy per page.
resource being examined
and/or to the material in
the textbook.
Discrepancies are
missed.
More than one
inaccuracy per
page.
Precision/
Logic
Statements are at the
best level of
information that
answers the question-not too vague but
not "lost in the
details" and missing
the big picture.
Statements are
mutually supporting
and follow from one
another. Any
contradictions are
explained.
Most statements are at
the best level of
information that answers
the question--not too
vague but not "lost in the
details" and missing the
big picture. Statements
are usually mutually
supporting and follow
from one another. Any
contradictions are
explained.
Fails to provide a
level of
information that
answers the
question--either
too vague or filled
with trivial details.
Statements are
internally
contradictory
without
explanation.
Problems in grammar,
spelling or style that
interfere with the author's
statements. (Multiple
problems in each
paragraph).
Statements are sometimes
on target and sometimes off
center. Segments of the
paper hang together but
other parts are unclear or
contradictory with no good
resolution.
Significant
problems in
grammar, spelling
or style that make
it challenging to
follow the
author's
statements.
4
Relevance/
Significance
Achieves the learning
objectives of the task.
Topic is significant to
both the course and
in larger senses (e.g.,
to individual, to the
region). Makes a case
for that significance.
Depth/Breadth Response displays a
full understanding of
the complexity of the
issue addressed and
multiple points of
view. Recognizes
varied interpretations
and implications.
No Plagiarism Rephrases in most
cases. Uses only
occasional quotation
that is in quotation
marks and is correctly
cited following APA
style.
Achieves the learning
Achieves learning
objectives of the task.
objectives minimally. Topic
Topic is obviously central choice is only vaguely
to the course.
related to the assignment.
Fails to achieve
the learning
objectives of the
task. The topic is
trivial or
irrelevant to
assignment.
Of the following 2 tasks,
does one well and the
other partially or does
both partially
a) Recognizing varied
points of view
b) Exploring the topic in
depth from one point of
view.
Response is both
narrower than
appropriate and
superficial. Fails
to recognize
varied
interpretations
and implications
of topic.
Of the following 2 tasks,
does one well and the other
not at all or does both
minimally
a) Recognizing varied points
of view
b) Exploring the topic in
depth from one point of
view.
Quotes properly but
Plagiarized. Used
excessive use of quotations. more than 3
consecutive
words from a
source without
quotation marks.
5
Example 3:
Rubric for scoring writing (2)
Source: Michael Theall, Youngstown State University, USA
Introduction
This is a generic rubric that can be used for holistic grading of any general writing assignment.
Rubric
Holistic: Overall, does the essay hold together as a piece of college-level writing?
DESCRIPTORS/SCORES: ACCEPTABLE
4. The essay shows adequate structure, support for major points, and command of the language
5. and may show quite good structure, support for all points, and good command of the language
6. and shows some insight, creativity, and very good command of the language.
DESCRIPTORS/SCORES: UNACCEPTABLE
3. The essay may wander a bit, fail to support points adequately, or may show unacceptable command
of some sentence-level features
2. and may show weakness in several of the areas, or serious weakness in one or more areas
1. and may show seriously flawed writing in most areas.
Focus: Does the writer follow directions and address the topic adequately?
DESCRIPTORS/SCORES: ACCEPTABLE
4. Follows directions, addresses the topic, and lets the reader know what's coming
5. and shows some insight into the topic
6. and even shows some creativity.
DESCRIPTORS/SCORES: UNACCEPTABLE
3. Drops a minor part of the directions or misses a point or stance central to the topic
2. and loses touch with the topic at times
1. and may not follow directions or may lose touch with the topic entirely.
Validity: Does the writer use accurate, non-trivial evidence and logic as support?
DESCRIPTORS/SCORES: ACCEPTABLE
4. The evidence is reasonably accurate and non-trivial
5. and may show an above-average range of knowledge of the topic
6
Example 4:
Rubric for scoring student presentations (1)
Source: Norback & Utschig Presentation Scoring System, University of Georgia
Introduction
This is part of a much larger rubric for evaluating student presentations. The complete rubric has four
components:




Customizing to the audience (shown below)
Telling the story
Displaying key information
Delivering the presentation.
Each component is described in much greater detail in the final full rubric. Some examples of behavioral
descriptions of a ‘wow’ performance are shown below.
Rubric: Customizing to the audience
Audience member characteristics are identified ahead of the presentation as observed through presentation
details tailored to audience needs and interests."
No
Connects with the audience, referring directly to their needs
Uses appropriate language for audience, describing concepts
at just the right level
Provides relevant details familiar to audience, using concrete
examples where relevant
Takes questions adeptly and answers them satisfactorily
Yes,
but
3
3
Yes
Wow!
1
1
Not
much
2
2
4
4
5
5
1
2
3
4
5
1
2
3
4
5
Examples of behavioral descriptors of a "wow!" level of performance.



"Greets audience and uses names of audience members if possible."
"Demonstrates that audience questions have been anticipated."
"Describes how skills/knowledge/experience was important in the work behind each key point."
7
Example 5:
Rubric for scoring student presentations (2)
Source: Marilla Svinicki, University of Texas at Austin College of Education, USA
Introduction
This rubric was developed for scoring a presentation made by graduate students in a course about applying
educational theories to special populations.
Rubric
Circle the level for each component and add weighted scores
I. Clear description of population characteristics and challenges (40% weight)
0
Not satisfactory
Incorrect, unclear or incomplete description
1
2
Satisfactory
Complete, correct and clearly presented for both
3
4
Very well done
Complete, correct and clearly presented, but shows an appreciation of connections
between population and challenges; OR goes beyond clear description to very
sophistication presentation or connection
II. Reasonable choice and description of theories that would be useful and why others are not useful (40%
weight)
0
Not satisfactory
Incorrect, unclear or incomplete choice, analysis, and/or description of theories in
relation to topic
1
2
Satisfactory
Correct theories chosen and tied to population being analyzed; standard analysis of
usefulness
3
4
Very well done
Meets all standards of satisfactory, but raises the analysis and comparison of theories
to professional level of sophistication
III. Professional and Engaging (20% weight)
0
Not satisfactory
Not satisfactory Presentation too casual or too disconnected with poor support
materials and visuals. Doesn't follow directions
1
2
Satisfactory
Presentation smooth and relaxed with adequate visuals and support materials
3
4
Very well done
Presentation has an extra degree of variety and creativity while still making the main
points clearly and professionally
8
Example 6:
Rubric for scoring an exam question
Source: Michael Theall, Youngstown State University
Question
1. In your work as a teacher who is a “reflective practitioner”, you must constantly make decisions. Some of
these decisions allow you the time to investigate topics and make plans (as in designing a new course for a
future offering), and some allow no time at all (as in the need for you to act immediately when a classroom
situation arises). Please describe the foundations or principles upon which you make these decisions.
OR
2. In your classroom, you face students whose preparation, abilities, skills, motivations, individual differences,
and personal situations vary a great deal. Some are eager and able. Some are resistant and unskilled.
Some have potential that is masked or limited by factors beyond their (and perhaps your) control. You have
to teach them all. What do you do to try to succeed in this most challenging task?
For either question …
You can respond in several ways, but your response should touch on one or more of the following questions
(and/or other points/issues you feel to be important):
 What factors do you consider and what issues do you take into account?
 What kinds of information are important?
 What are your beliefs and motivations?
 When you have time, what process do you use to design effective instruction?
 When you must respond immediately, what guides your thinking and actions?
 What resources do you call upon?
 What human/interpersonal issues come into play?
 What theories and “best practices” do you rely on?
Complete answers to these question and sub-questions would fill a book. Do not try to put down everything you
know. Rather, indicate the important things that guide and influence your decisions. If you had ‘the’ answers,
you would become famous … but teaching and learning present unique situations and the most successful
teachers know there is no, one answer. They understand that they must reflect on a variety of issues and adapt
to the variety of students and situations they face. Reflect for us here. Tell us what do you do and why do you
do it?
Rubric follows on the next page.
9
RATINGS
SCORE
SKILLS
SUPERIOR = 4
STRONG = 3
ACCEPTABLE = 2
MARGINAL = 1
ABSENT
=0
Consider
important
factors in
successful
instruction
Thorough
outline of
important
factors
=4
Most factors
covered
=3
Some factors
covered
=2
Many factors
omitted
=1
Absent = 0
3
Incorporate
philosophy
of teaching &
beliefs
about teaching
& learning
Strong beliefs:
clear & relevant
to
tchng & lrng
=4
Strong beliefs
discussed
=3
Some beliefs
presented
=2
Little
philosophy or
few beliefs
included
=1
Absent = 0
3
Demonstrate
knowledge of
basic
instructional
design
ID process or
model(s)
discussed
in some depth
=4
Brief process or
model
description
=3
ID model(s) or
process
mentioned
=2
ID model(s) or
process
implied
=1
Absent = 0
2
Show concern
for
students &
learning
Clear, learnercentered
approach
=4
Some discussion of
l-c approach
=3
Reference(s) to
l-c approach
=2
Learnercenteredness
implied
=1
Absent = 0
4
Demonstrate
reflective
practice in
action
Shows good
understanding
of
reflective
practice
=4
Discusses reflective
practice
=3
Mentions
reflective
practice
=2
Reflective
practive
implied
=1
Absent = 0
3
Connect to
"best
practices" &
theories
Strong
connections to
theory &
practice
=4
Some connections
to
theory & practice
=3
Few
connections to
theory &
practice
=2
Theory &
practice
links implied
=1
Absent = 0
2
Points
>>>>>>>>>>>>
>>>>
Writing
10
SCORING****** PASS = > 12 pts.
RATER: M.
Theall
STUDENT
FAIL = < 13 pts.
deductions
>>>>>>
Total points
>>>>>>>>>>>>
GRADE
>>>>>>>>>>>>
>>>
Pass
11
Example 7:
Rubric for first component of proposal paper
Source: Marilla Svinicki, University of Texas at Austin College of Education
Note
This rubric was used for scoring an initial paper in a graduate class.
Purpose
To create an understanding of the problem and the need for a solution that is instruction-based or behavior
management based.
Questions
1. What is the problem for which you are proposing a solution?
2. Why is it important?
3. Why is an instructional or behavioral management solution called for?
Rubric
Basics
The paper must state the problem so that someone not in the area can understand what the
target problem looks like from a behavioral, learning, instructional or management standpoint (2
pages). It also makes a case for why this problem needs to be solved, including what impact it is
having on the individual or class of individuals who represent the target audience (2 pages).
Finally it relates the problem to the purpose of this class: the use of psychological theories to
design instruction to address the issue (1 page). I recommend providing headings to indicate the
subcomponents of the paper.
An ‘A’ paper
For this to be an A paper, the above must be very clearly and concisely written, with convincing
main arguments, and give the reader a clear sense of how solving this problem will benefit the
target audience. The paper must build a strong tie between the problem and basing the solution
on instructional or management theory or theories studied in this class. (For the last of these
criteria, you may look ahead to theories we have not discussed if they are appropriate.)
A ‘B’ paper
For this to be a B paper, at least two of the three basics listed above must be included, but the
writing or argumentation will be less clear, concise, or compelling than an A paper. There will be
fewer examples of key evidence for the seriousness of this problem and fewer reasons why the
problem needs to be resolved using an instructional or management solution. References to the
theory or theories that make up this course may be fewer.
A ‘C’ paper
A paper will receive a C if it is not clear what the problem being addressed is or if only one of the
three basics listed above is included. A C might also reflect a poorly written paper or one with a
lot of mistakes, either conceptual or mechanical.
12
Example 8:
Rubric for quizzes and homework in Environmental Science
Source: Diane Ebert-May and Tsao, Michigan State University
Note
This rubric can be used with either homework problems or on short quizzes.
Rubric
Level of
Achievement
Exemplary
General Approach
•
•
•
•
Adequate
•
•
•
•
Needs Improvement
•
•
•
•

Comprehension
Addresses the question.
States a relevant, justifiable answer.
Presents arguments in a logical
order.
Uses acceptable style and grammar
(no errors).
•
Does not address the question
explicitly, although does so
tangentially.
States a relevant and justifiable
answer.
Presents arguments in a logical
order.
Uses acceptable style and grammar
(one error).
•
Does not address the question.
States no relevant answers
Indicates misconceptions.
Is not clearly or logically organized.
Fails to use acceptable style and
grammar (two or more errors).
•
•

•


Demonstrates an accurate and
complete understanding of the
question.
Backs conclusions with data and
warrants.
Uses 2 or more ideas, examples
and/or arguments that support the
answer.
Demonstrates accurate but only
adequate understanding of question
because does not back conclusions
with warrants and data.
Uses only one idea to support the
answer.
Less thorough than above.
Does not demonstrate accurate
understanding of the question.
Does not provide evidence to support
their answer to the question.
No Answer
13
Example 9:
Rubric for evaluating senior design projects in Chemical Engineering
Source: Ron Miller, Colorado School of Mines
Introduction
This rubric is used to evaluate very extensive complex projects that require students to produce original
designs for senior level courses intended as capstone courses. Note that the rubric also relates each
objective being evaluated in the left most column.
Outcome
ChE graduates
will be able to
design chemical
engineering
processes or
systems which
meet specified
requirements by:
a. identifying
specific project
objectives based
on general
project and client
requirements
b. gathering and
using relevant
background
information
c. generating
and analyzing
alternative
solutions by
synthesizing and
applying
appropriate
chemical
engineering
knowledge
d. choosing the
optimal solution
based on
evaluation of
technical and
economic criteria
4 - Exemplary
3 - Proficient
2 - Apprentice
1 - Novice
All important
project objectives
are identified
Important
objectives are
identified but 1 or
2 minor ones are
missing.
Most objectives
are identified but
at least 1 or 2
important ones
are missing.
Most or all
important
objectives are not
identified.
All relevant
information is
obtained and
used to support
design.
Three or more
alternative
solutions are
considered; each
is correctly
analyzed for
technical
feasibility.
Sufficient
information is
obtained and
used to support
design.
At least 3
alternative
solutions are
considered;
analysis is
complete but
contains minor
procedural errors.
Some information
is obtained, but
more is needed to
support design.
No significant
background
information is
gathered.
At least 2
alternative
solutions are
considered;
analysis contains
minor conceptual
and/or
procedural errors.
Only one solution
is recommended;
analysis does not
apply all relevant
chemical
engineering
knowledge.
Best solution is
recommended
based on stated
criteria.
Reasonable
solution is
recommended;
other alternatives
should have been
developed
Satisfactory
solution is
recommended;
better solutions
were available to
be considered.
Only solution
considered; no
optimization
included; better
solutions were
avaialble.
Score
14
ChE graduates
will be able to
analyze the
economic
profitability of
chemical
engineering
projects or
systems
ChE graduates
will demonstrate
an ability to
communicate
effectively in
writing.
Economic analysis
is complete and
correct; all
relevant
economic factors
are considered.
Economic analysis
omits 1 or 2
minor economic
factors but
computations are
correct.
Economic analysis
omits 1 or more
major economic
factors, but
computations are
correct.
Economic analysis
contains major
computational
errors and/or
omits more than
1 major economic
factor.
Written report is
virtually errorfree, logically
presents design
recommendations
and analysis, is
well-organized
and easy to read
and contains high
quality graphics.
Written report
presents design
recommendations
and analysis
logically, s well
organized and
easy to read,
contains high
quality graphics
and contains few
minor
grammatical or
rhetorical errors.
Written report is
generally well
written but
contains some
grammatical,
rhetorical and/or
organizational
errors; design
recommendations
and analysis are
mentioned but
not fully
discussed.
Written report
does not present
design
recommendations
or analyses
clearly, is poorly
organized and/or
contains major
grammatical
and/or rhetorical
errors.
15
Example 10:
Rubric for problem-based learning discussion lab
Source: Source: Dr. Patrick Davis, University of Texas at Austin College of Pharmacy, USA
Student 9
Student 8
Student 7
Student 6
These are possible concepts that may fall under these
categories. Please use them globally, not as a checklist.
Preparation & Clinical Skills (6 points)
 prepared and reviewed multiple resources
 appeared able to answer questions without
hesitation
 defined terms and clarified concepts ahead of
time
 volunteered meaningful answers throughout
session
 asked relevant questions throughout session
 used appropriate vocabulary/pronunciation
Group Skills & Communication Skills (6 points)
 encouraged others to contribute
 shared ideas with others
 exhibited tolerance and respect for others
 was assertive, confident (without dominating)
 was clear and concise
 analyzed contributions of others
Administration (3 points)
 dressed appropriately for lab as per syllabus
 was professional and enthusiastic
 was punctual
Student 5
Date:
Student 4
Lab Day (circle one): M T W Th F
Student 3
Case #:
Student 2
Facilitator:
Student 1
Introduction
This rubric is used to evaluate student participation in a problem-based learning discussion laboratory
session on a Pharmacotherapeutics course.
Total Score (15 points possible):
Please use the space below to write any comments (positive or negative) about individual students:
16
Example 11:
Using rubrics to help students develop their own ability to evaluate
their work
Source: John Cowan, Napier University
Introduction
The following example shows rubrics can be used to help students learn how to make evaluative
judgements of their own work. Shown below is one example of a class activity he uses in this manner.
Matching performance to evaluative criteria
Purpose: To engage each student in objectively and formatively evaluating their own performance against
familiar criteria, which have previously been applied to similar work, but in that case by a grader or tutor.
Outline
a. This activity is based on a current task. It has been used, for example, in conjunction with a
discursive essay in social sciences, a first year engineering design; and a reflective journal featuring
a recent critical incident on a placement.
b. On completion of the task, students are immediately offered a podcast guiding them through the
process of evaluating their work, leading to an accumulation of suggestions for both encouraging
and critical comments, as feedforward.
c. Students should now assemble and summarise the elements in their self-judgement, the data on
which it was based, and their evaluative judgements.
d. Marking tutors, acting virtually as auditors at this stage, quickly check the evaluations, and return
(without judgements) with comments only when the judgement has overlooked something
significant, or appears unjustifiably generous – or harsh.
e. Assessment, in the form of a mark or grade, is not declared or (preferably) negotiated (as per
Taras, 2001), until all of the above steps have been completed.
f. The tutor informs the student of the outcome of their auditing of the student’s self-judgement.
The common outcome is that each student will have had a relevant and useful experience of identifying
personally-related data, and matching it to multiple criteria, in an objective process. The impact on the
students’ further development is strengthened by the immediacy of the details of feedback and
feedforward, and the lack of emphasis on the mark or grade to be awarded.
17
Download