Rubrics and Other Scoring
Methods
Sue Brookhart
January 26, 2015
Introductions
• Sue Brookhart, Ph.D.
• Juliette Lyons-Thomas, Ph.D. (Fellow, Regents
Research Fund)
2
Webinar Norms
• All phones will be placed on mute
• If you have a question, you can type into the
chat box, and your question will be addressed
during a break

The chat box icon is located at the top right hand
corner of your screen (remember to direct your
chat to “Everyone”)
• At the end of the webinar, you will be asked to
fill out a survey based on your experience
today
3
Learning Outcomes
• Participants will learn to create (or select and
adapt rubrics) that match intended learning
outcomes.
• Participants will learn to check for consistency
among scorers.
• Participants will learn to use checklists, rating
scales, and point schemes for special
purposes.
4
Performance Assessment
• Task(s) or assignment must tap the student
learning outcome



Knowledge
Skills
Level of Thinking
• Rubrics or other scoring scheme must
accurately describe student performance
along a continuum of quality
5
Criteria
• Important for all scoring methods

Rubrics, Checklists, Rating Scales, Point Schemes
• These are what connect student performance to
the learning outcomes they indicate
6
RubricsCriteria and Performance
Level Descriptions
Criteria: What qualities need to be in the work?
Performance level descriptions: How well does
the work exhibit these qualities?
7
Example – Choral Music Rubric
Accuracy
Text
Part Work
Tone Quality
Interpretation
A
Always on pitch
Rhythms correct
B
Only a few incorrect
notes
Mostly accurate
rhythm, stable beat
C
Usually on pitch
Some minor rhythm
problems, generally
a steady beat
Fully memorized
Always clear with
precise sounding
consonants
Knows all parts
Well balanced
sound (both parts
heard equally)
Clear and projected
Even tone between
head & chest voice
Consistent and
effective breath
support
Mostly memorized
Mostly clear with
attempted ending
consonants
Knows most parts
Mostly balanced
sound
Some
memorization
Usually clear but
missing consonants
One part not well
known
Adequate balance
between parts
Beginning to be
clear & projected
Much difference in
tone between head
& chest voice
More breath
support needed
Consistent and
correct use of
dynamics and
expression
Mostly clear &
projected
Some difference in
tone between head
& chest voice
Breath support
effective most of
the time
Effective use of
dynamics and
expression
Some dynamic
variation
Adequate
expression
D
Rarely on pitch,
many incorrect
notes
Some rhythmic
errors and changes
in beat
Little memorization
Unclear or
mumbled
F
Not on pitch
Many rhythmic
errors and changes
in beat
Both parts not well
known
Parts not balanced
Unclear and
unsupported (lack
of projection), or a
harsh tone in many
sections
Parts mostly
unknown
Not able to hear
two parts
Weak (no
projection)
Tone harsh,
unsupported,
and/or uncontrolled
Very few dynamics
Little expression
No dynamics
No expression
No evidence of
memorization
Not clear
8
A Counter-Example - What happens
when bad rubrics are applied to good work?
9
Student A – 7th grade inclusion
10
11
Student B – 7th grade inclusion
12
13
Sometimes rubrics can be very
concrete
• Example – third grade
• Concepts of Multiplication poster
• Present a multiplication problem, a repeated
addition problem, a picture with equal groups,
and an array for the same multiplication fact
• Poster
14
15
Example
16
17
Example
18
More Complex Rubrics for
More Complex Tasks
• Several criteria


Analytic rubrics – criteria considered one at a
time
Holistic rubrics – criteria considered
simultaneously
• Criteria should be qualities that work should
exhibit (e.g., position is supported, steps are
explained with mathematical vocabulary, cites
appropriate sources)
• Criteria should NOT be about directions (e.g.,
has 4 sentences, cites 3 sources)
19
Example
2
1
0
Understanding Complete
the Problem
understanding of
the problem
Part of the problem Complete
misunderstood or
misunderstanding
misinterpreted
of the problem
Planning a
Solution
Plan could have
led to a correct
solution if
implemented
properly
Partially correct
plan based on part
of the problem
being interpreted
correctly
No attempt, or
totally
inappropriate plan
Getting an
Answer
Correct answer
and correct label
for the answer
Copying error,
computational
error, partial
answer for a
problem with
multiple answers
No answer, or
wrong answer
based on an
inappropriate plan
(NCTM, 1987, 1991)
20
Example
Jan's Snack Shop has 3 flavors of ice cream—vanilla, chocolate,
and strawberry. The ice cream can be served in a dish, a sugar
cone, or a regular cone.
There are 9 people who choose 1 dip of ice cream in a dish, or in
a sugar cone, or in a regular cone, and all of their choices are
different. List or show the 9 different choices.
Could another person have a choice that is different from one of
these 9 choices? Why or why not?
(NAEP 2003, grade 4,
Block M10 #17)
21
Example to Score
22
Example to Score
23
Example to Score
24
Consider a Template
4
Task is complete and correct, plus it includes an extension,
exceptional insight, or some other feature that is “above and
beyond” what was required or intended.
3
Task is complete and correct. Response shows understanding of
intended learning outcome.
2
Task is partly complete and correct. Response shows some
understanding of intended learning outcome.
1
Task is not complete or correct. Response shows little/no
understanding of intended learning outcome.
NOTE: DON’T use this “rubric” as is – it’s a template. It lacks CRITERIA.
Replace the “boilerplate” language with criteria specific to your
learning outcome of interest.
25
Example – 6th ELA – Writing a well-structured
paragraph in response to a prompt
• Direct instruction and practice, four weeks
• Learning outcome = can write a paragraph that




Answers the prompt question
Has a focus/topic sentence
Has detail sentences that relate to the focus
Has an effective ending sentence
26
Example – 6th grade writing
4
Task has a focus sentence, details, and conclusion, plus it
includes a particularly compelling response to the prompt, for
example with especially vivid details.
3
Task is has a focus sentence, details, and conclusion.
2
Task is has some attempt at a focus sentence, details, and
conclusion.
1
Task does not have a focus sentence, details, and conclusion.
27
Example – 6th grade writing
28
Example – 6th grade writing
29
For Simple Tasks
3
Task is complete and correct. Response shows understanding of
intended learning outcome.
2
Task is partly complete and correct. Response shows some
understanding of intended learning outcome.
1
Task is not complete or correct. Response shows little/no
understanding of intended learning outcome.
NOTE: DON’T use this “rubric” as is – it’s a template. It lacks CRITERIA.
Replace the “boilerplate” language with criteria specific to your
learning outcome of interest.
30
Kindergarten Example
Making Equal Sets
3
Elements of each set are organized in a single line. There is a
one-to-one correspondence between elements of each set.
2
There is an attempt at organizing elements of each set and
showing correspondence between elements of each set. These
attempts are not entirely successful.
1
Elements of each set are not organized. No correspondence is
shown between elements of each set.
31
Kindergarten Example
Making Equal Sets
A
B
32
Ensuring Scoring Consistency
• Use a clear scoring guide (e.g., rubric)
• Score one question/task at a time
• Score content separately from other factors
(e.g., grammar, spelling)
• Score questions/tasks anonymously
33
Ensuring Scoring Consistency
• Select a few papers (questions, tasks) from
different classes
• Double-score this set of papers (questions,
tasks)
34
A Tale of 8th Grade ELA Portfolios
35
Checklists
• Most useful for self or peer assessment of
following directions for the assignment
• Useful for basic skills (e.g., capital and period
for each sentence) or concrete behaviors (e.g.,
checklist for washing hands) and procedures
(e.g., making a wet mount slide).
36
Example – 3rd grade learning support
37
38
Example – Checklist for preparing a
wet mount slide
 Assemble materials (slides, cover slips, pipet, sample and any
solutions or chemicals)
 Place a drop of the sample or staining solution in the center of a
clean slide
 Add cells to be viewed under the microscope (unless they are
already in the sample)
 Angle cover slip over the sample with one edge touching the
slide
 Lower cover slip gently, to avoid bubbles in the solution
 Remove any excess solution by touching a paper towel to the
edge of the cover slip
 Place slide on the stage of a microscope and view at low power
to check slide contents
39
Checklists and Scoring
• Typically, checklists are NOT scored, but used
for feedback.
• When a score is needed for a procedure that
can be governed by a checklist, it is possible
to use the checklist with 1-0 scoring (check = 1
point, no check = 0 points).
• Note that using a checklist assumes that the
only measure of “quality” is that something
was done – not how well – and this is why
rubrics are usually preferred for scoring.
40
Rating Scales
• Rating scales are useful for behavior, learning
skills, and citizenship.

Frequency scales (e.g., “always, frequently,
sometimes, never”) are usually better than
evaluative scales (e.g., “excellent, good, fair,
poor”)
• Rating scales are useful for measuring
attitudes, interests, and other survey-type
questions.
• Not recommended for scoring student school
work – can be useful for self-assessment
41
Example
High school
Chemistry
42
Point Schemes
• Best used to score brief, focused constructed
response test questions


Essay questions
Show-the-work math problems
• Need a new point scheme for every question
• Cannot share with students
43
Example
Fill in the chart below with the name of each of the
three branches of government and the main purpose
of each branch.
Brand of Government
Main Purpose
Total possible points = 6
1 point each for naming Executive, Legislative, & Judicial
1 point each for main purpose, at minimum: Enforces laws
(Exec), Makes laws (Legislative), Interprets laws (Judicial)
44
Example
[NAEP, Grade 8, 2005; Block S13 #10]
Some scientists think that the Earth's climate is getting warmer. If
these scientists are correct and the Earth keeps getting warmer
for the next 50 years, what will happen to the oceans? Explain
why this would happen. [space given for answer]
If these scientists are correct, what things about the Earth's
weather will change? Explain why this would happen. [space
given for answer]
If these scientists are correct, what will happen to plants? Explain
why this would happen. [space given for answer]
Scoring: Response consists of six correct parts: a prediction and
an explanation for each of oceans, weather, and plants.
45
Summary
• Use rubrics to match LEARNING OUTCOMES,
not directions.
• Use other scoring
methods for special
purposes.
• Check consistency for
any scoring that has consequences for
students, teachers, or schools
46
Thank You
• The slides and a video of this webinar will be
posted at
https://www.engageny.org/resource/teachingcore-assessment-literacy-series-materials
• Please provide feedback on your webinar
experience, or suggestions for future topics,
by filling out this survey:
https://www.surveymonkey.com/s/rubricswebi
nar
47