Assessing a Skill or Performance
Focus on Learning: Part 1
August 2012---Sandy Odrowski
Session Outcomes
During this workshop you will



Identify tools that can be used with performance assessments,
Distinguish between rubrics, checklists and rating scales,
Explore and share tips to guide the assessment of performances.
Key Assessment Concepts and Definitions
What is Performance/Authentic Assessment?
Performance Assessment – close proximity to “actual criterion situation”. Usually
measures complex skills, cognitive processes and communication important in real
world (contextualized tasks) (Palm, 2008).
Authentic Assessments –defining features are the specific cognitive processes
(disciplined inquiry) and products (knowledge beyond the mere reproduction of
presented knowledge) considered important in the perspective of life beyond school
(Newman & Archibald, 1992).
Assessment vs. Evaluation

Lots of controversy/confusion over these terms
Assessment = feedback on practice; supports learning but may or may not
generate marks/grades
Evaluation= summary measurement providing some kind of grade/mark/final
feedback
Validity
Validity refers to the degree to which an assessment measures the intended
learning outcome.
Or more simply stated…
Does the test measure what it is intended to measure? (the learning outcome?)
1
Reliability
Reliability refers to the consistency of test scores. Reliability affected by testing
conditions, rater variability.
Or more simply stated…
Does the test produce consistent results?
Determining the Purpose of Assessment
In order to prepare a good assessment you will need to first identify a clearly
defined purpose (Brualdi1998).
 What is the learning outcome to be assessed?
 What should the learner be able to do?
 At what level should the student be performing?
Who is the “Assessor”?




Multiple perspectives
Self
Peer
Teacher
2
Selecting an Assessment Tool
Assessment
Tool
Attributes
Checklist
•
•
•
Performance/
•
Rating Scale
•
Rubric
•
•
•
•
Sample
the least complex form of scoring
looking for the
presence/absence of specific
elements in the product of a
performance
Useful for assessing simple
performances or achievement in
which the individual elements
being assessed typically involve
dichotomous types of judgments.
unlike the typical checklist, it
allows for attaching some quality
to elements in the process or
product
ratings can be numeric or
descriptive
multiple criteria are applied to
the assessment
quality of the performance &
product are typically transparent
in the assessment criteria
appropriate for complex tasks
rather than discrete activities
Rubrics are designed to allow for
the differentiation between levels
of achievement, or development,
by communicating detailed
information about what
constitutes excellence.
3
Participates in group problem solving:




4
3
2
1
(Outstanding)
(Satisfactory)
(Tolerable)
(Unsatisfactory)
Selecting a Performance Task
Assessment task (performance, product or process) determined to reflect learning
outcome.
Assessment task attributes – essential elements or characteristics of a good
performance of an assessment task
Performance criteria (descriptors) – specific behavioural descriptions of
performance at each level of performance
A Checklist for Choosing Performance Tasks
Does the task match the expected learning (learning outcome or course
learning requirement)?
Does the task adequately represent and elicit the content and skills you
expect the student to attain?
Does the task enable students to demonstrate their capabilities and progress?
Does the assessment use “authentic”, real world tasks?
Does the task require the learner to integrate their learning?
Can the task be structured to provide a measure of several outcomes?
Does the task match an important outcome which reflects complex thinking
skills?
Does the task pose an enduring problem type-- the type the learner is likely
to encounter in the future?
Is the task fair and free of bias?
Will the task be seen as meaningful by important stakeholders?
Will the task be meaningful and engaging to students so that they will be
motivated to show their capabilities?
4
Yes
No
Steps in Creating a Performance Tool
Steps in creating a performance assessment tools (critical to identify the
performance attributes and criteria)
1. Identify the learning outcome and the purpose of the assessment
2. Select an authentic task (or product creation) that matches the learning
outcome
3. Identify attributes for the task (or product creation)
4. Identify the descriptors (performance indicators) for each attribute. Describe
above average, average and below average performance for each descriptor
5. Create the performance assessment tool (checklist, rating scale, rubric)
Tips for Developing/Improving Performance Assessment
Tools








Review the learning outcome and purpose of the assessment tool
(validity)
Collect samples of student work that exemplify levels of performance
(validity)
Consult with experts in fields to develop “authentic performance tests”
and to validate task attributes and performance descriptors (validity,
authenticity)
Develop performance scoring instruments collaboratively with
colleagues
Gather multiple perspectives on scoring of same performance (rater
reliability)
Share scoring rubric with students in advance of the performance task
(equity, transparency, accountability)
Share performance exemplars (sample projects, assignments, video
depictions) with students in advance of performance task
(transparency, accountability)
Be prepared to review and revise assessment tool (accountability,
validity)
5
Learning Activity: Creating, Administering and Evaluating
Performance Assessment Tools.
Group Activity: collaboratively create, administer and evaluate a performance
assessment tool for the assigned task:

Accurately measure and report a radial pulse.

Design, construct and fly a paper airplane.
Assign group roles (Insert name in the space provided)
Performance expert ____________________________
Student ____________________________________
Peer (reporter)_______________________________
Teacher (timekeeper)___________________________
Complete the following Steps:

In consultation with the performance expert, determine performance
attributes and criteria (what you feel is essential to achieve the learning
outcome) for the identified task.

Collaboratively develop the assessment tool (checklist, rating scale or
rubric) using template provided.

The performance expert instructs the student on the performance task
using the assessment tool as a guide.

The student completes a “return demonstration” of the performance
task.

The teacher and peer complete the developed assessment tool based
on the observed student’s performance.

Discuss the advantages and disadvantages of the developed assessment
tool from a variety of perspectives (student, peer and teacher). (use next
page)
6

Be prepared to share your group’s finding with the larger group.
Type of Assessment
tool
Advantages
Rubric
Checklist
Performance Scale
7
Disadvantages
Review: Mix/Match
Instructions: Record the correct number for the correct definition in the
space provided
1
Learning Outcome
___
the least complex form of scoring
looking for the presence/absence of specific
elements in the product of a performance
2
Performance/Authentic
Assessment
___
multiple criteria are applied to the assessment
quality of the performance & product are typically
transparent in the assessment criteria
3
Rubric
___
unlike the typical checklist, it allows for attaching
some quality to elements in the process or
product
ratings can be numeric or descriptive
4
Performance scale
___
feedback on learning/ practice; supports learning
but may or may not generate marks/grades
5
Checklist
___
6
Validity
___
Conducted at the end of the instructional periodusually a summary of learning assessment and
involves marks/grading.
is a form of assessment that requires students to
perform a real life task rather than select an
answer from a ready-made list.
7
Reliability
___
The culminating performance the student should
be able to do at the end of a course or session.
8
Summative
assessment
___
refers to the degree to which an assessment
measures the intended learning outcome.
9
Formative assessment
___
Refers to the consistency of test scores. Can be
affected by testing conditions, rater variability.
8