®
Worldwide Instructional Design System
Performance Assessment
in
Online Learning
Innovations 2005
Judy Neill
© 2003-2005 WIDS-WTCSF
neillj@wids.org
Wisconsin Technical College System Foundation, Inc.
Assessment in Online Learning
What types of assessment strategies are you
currently using in online learning?
© 2003-2005 WIDS-WTCSF
Performance Assessment
Assumption
As a result of
learning, learners
should be able to
do something new
or do something
better than they
could prior to the
learning
experience.
© 2003-2005 WIDS-WTCSF
Performance Assessment
Assumption
Do Means:
Produce a product
Perform a service
Carry out a performance
Make a decision
Solve a problem
Draw a conclusion
© 2003-2005 WIDS-WTCSF
Assessment of Learning
The process of measuring
performance according to
predetermined outcomes and
established criteria.
© 2003-2005 WIDS-WTCSF
Assessment of Learning
Show me the learning!
© 2003-2005 WIDS-WTCSF
The Purposes of Assessment
 Continually improve learning and
teaching
 Document learning
 Credential learner
© 2003-2005 WIDS-WTCSF
Credentialing Performance
 Listing
 Naming
 Defining
 Describing
 Repeating
 Recognizing
© 2003-2005 WIDS-WTCSF
Credentialing Performance
Produce a product
Perform a service
Carry out a performance
Make a decision
Apply
knowledge
Solve a problem
Draw a conclusion
© 2003-2005 WIDS-WTCSF
Assessment in Online Learning Today
Even colleges that are working to implement learningcentered expectations and practices may throw these
principles to the wind when they take on the challenges
of online learning. . .
© 2003-2005 WIDS-WTCSF
Assessment in Online Learning Today
. . . taking shelter behind
traditional, more
comfortable practice of
relying on“telling and
testing as teaching.”
© 2003-2005 WIDS-WTCSF
Assessment in Online Learning Today
 Efforts focus on selecting and
implementing a delivery platform
and preparation of content for online
delivery.
 Popular delivery platforms provide
easy-to-use tools that emphasize
objective paper/pencil type testing.
 Many students perceive multiple
choice tests as the easy way out.
 Teachers are juggling time
constraints with the need to adapt to
new technologies.
© 2003-2005 WIDS-WTCSF
The Assessment Process
1. Establish learning outcomes that meet learners’
performance needs
2. Determine whether or not learners have
achieved the outcomes
3. Reflect on the results for continually improving
learning and teaching
4. Move to next level of learning or intervene to
improve learning/teaching
5. Document assessment of learning
© 2003-2005 WIDS-WTCSF
The assessment process is integrated
with the learning process.
Who?
What?
When?
How?
© 2003-2005 WIDS-WTCSF
Learners
Exit Outcomes
Competencies
Identify
learning needs.
Teach the right
thing.
Assessment
Assess what
you intended to
teach.
Learning
Activities
Teach what you
intend to
measure.
Three of the four questions are
assessment questions.
Who?
What?
When?
How?
© 2003-2005 WIDS-WTCSF
Learners
Exit Outcomes
Competencies
Assessment
Learning
Activities
What are the
learners’ needs?.
What will they
learn to do?
How will we know
when they can do
it?
How will they
learn?
Performance Assessment
Requires performance of one
or more target outcomes in as
close to an authentic situation
as possible.
© 2003-2005 WIDS-WTCSF
What makes assessment authentic?
Assessments that most closely
relate to, or mirror, real-life
application are most authentic.
© 2003-2005 WIDS-WTCSF
Why is authentic assessment valuable?
“The proof is in the pudding. . .”
Plum Pudding
one pound of raisins stoned
one pound of currants
half a pound of beef-suet
quarter of a pound of sugar two
spoonfuls of flour three eggs
a cup of sweetmeats
a wineglass of brandy
Mix well, and boil in a mould
eight hours.
© 2003-2005 WIDS-WTCSF
Why is authentic assessment valuable?
“. . . and in the eating there of.”
 What does it look like?
 How does it smell?
 What is its texture?
 Is the temperature
right?
 How does it taste?
© 2003-2005 WIDS-WTCSF
Why is authentic assessment valuable?
When learning/assessment relate closely to
real-life, learning is enriched because:
 Learners become more engaged.
 Learners see the purpose of what they
are learning.
 Learners make connections between
what they are learning and what they
already have learned.
© 2003-2005 WIDS-WTCSF
Why is authentic assessment valuable?
When learning/assessment relate closely to
real-life, learning is enriched because:
 Learning moves beyond memorization
and recitation to application, analysis,
synthesis, and evaluation.
 Learners are better prepared to apply or
transfer what they learn to performing
real life tasks and solving real life
problems.
© 2003-2005 WIDS-WTCSF
Levels of Authenticity
Authenticity is a continuum,
however for discussion purposes
let’s break it down into five levels:
Level V
Authentic Assessment
Level IV
Simulation/Role-Play
Level III
Case Study or Scenario Application
Level II
.
Minimally Authentic
Level I
Non-Authentic
© 2003-2005 WIDS-WTCSF
Level 1: Non-Authentic Assessment
Requires learners to solve or answer theoretical
or abstract questions or problems with little or
no connection to authentic application.
Examples:
Learners answer questions that require
them to recall information.
Learners solve math or science problems
without context or application.
Level V
Authentic Assessment
Level IV
Simulation/Role-Play
Level III
Case Study or Scenario Application
Level II
.
Minimally Authentic
© 2003-2005 WIDS-WTCSF
Level I
Non-Authentic
Level 2: Minimally Authentic Assessment
Requires learners to answer questions or
solve problems that reference realistic data
or information, but do not reflect a cohesive
work or personal life experience.
Example:
 Learners describe the components of
a typical marketing plan in response to
a completion, multiple choice or essay
question.
Level V
Authentic Assessment
Level IV
Simulation/Role-Play
Level III
Case Study or Scenario Application
Level II
.
Minimally Authentic
© 2003-2005 WIDS-WTCSF
Level I
Non-Authentic
Level 3: Case Study or Scenario Assessment
Requires learners to solve a problem in response to a
case study that includes the kind of tools, data, and
information that would be available in a real-life
setting.
May require learners to use planning or conceptual
tools, such as a flowcharts or spreadsheets, to
perform higher-level cognitive tasks like analysis or
evaluation.
Examples:
 Learners develop an emergency response plan
based on a case study situation.
Level V
 Learners respond to scenario-based questions
Authentic Assessment
on a Q&A test.
Level IV
Simulation/Role-Play
Level III
Case Study or Scenario Application
Level II
.
Minimally Authentic
© 2003-2005 WIDS-WTCSF
Level I
Non-Authentic
Level 4: Simulation/Role Play Performance
Assessment
Requires learners to perform target
outcome(s) in a simulated work or personal
life situation.
Examples:
Learners role play conflict resolution
strategies in a simulated workplace
setting.
Learners make critical decisions in a
simulated weather emergency.
Level V
Authentic Assessment
Level IV
Simulation/Role-Play
Level III
Case Study or Scenario Application
Level II
.
Minimally Authentic
© 2003-2005 WIDS-WTCSF
Level I
Non-Authentic
Level 5: Authentic Performance Assessment
Requires learners to perform target
outcome(s) in an authentic setting.
Example:
 Learners develop a marketing
plan for a local business that
they identify.
Level V
Authentic Assessment
Level IV
Simulation/Role-Play
Level III
Case Study or Scenario Application
Level II
.
Minimally Authentic
© 2003-2005 WIDS-WTCSF
Level I
Non-Authentic
How can we effectively facilitate higher level
assessments online?
 Strategy 1: Strive for authenticity.
 Strategy 2: Provide learners with the tools and
information they need.
 Strategy 3: Create an online learning environment
that supports higher level assessment.
 Strategy 4: Provide opportunities for practice and
feedback.
© 2003-2005 WIDS-WTCSF
Strategy 1: Strive for authenticity
 Set up an authentic scenario such
as a simulated organization or
planning/project team.
 Have learners play the role of
various organization/team members.
© 2003-2005 WIDS-WTCSF
Strategy 1: Strive for authenticity
 Follow authentic processes for
planning, research, decisionmaking, product development,
communication, and
documentation.
 Use authentic documents, forms,
data, and evaluation criteria if
available.
 Consider the levels of authenticity.
© 2003-2005 WIDS-WTCSF
Strategy 2: Provide learners with the tools
and information they need
 Provide complete detailed directions in the
online environment.
 Use scoring guides to communicate
performance standards.
 Provide samples and examples of exemplary
products/performances.
 Use online communication tools to facilitate the
collaborative development of products and
performance.
© 2003-2005 WIDS-WTCSF
Strategy 2: Provide learners with the tools
and information they need
 Provide time for teambuilding if learners are
expected to work in groups to complete
projects/performances.
 Provide necessary data and information or
clearly direct students to where they can
obtain data and information.
 Assist learners in developing a realistic
timeline for completing project/performance.
 Incorporate assessment objects (learning
objects with interaction and feedback).
© 2003-2005 WIDS-WTCSF
Tools and Information: Learning Plans
© 2003-2005 WIDS-WTCSF
Online learning components were developed in WIDS.
Tools and Information: Learning Plans
© 2003-2005 WIDS-WTCSF
Online learning components were developed in WIDS.
Tools and Information: Learning Plans
© 2003-2005 WIDS-WTCSF
Online learning components were developed in WIDS.
Tools and Information: Learning Plans
© 2003-2005 WIDS-WTCSF
Online learning components were developed in WIDS.
Tools and Information: Performance
Assessment Task
© 2003-2005 WIDS-WTCSF
Online learning components were developed in WIDS.
Tools and Information: Performance
Assessment Task
© 2003-2005 WIDS-WTCSF
Online learning components were developed in WIDS.
Tools and Information: Performance
Assessment Task
© 2003-2005 WIDS-WTCSF
Online learning components were developed in WIDS.
Tools and Information: Performance
Assessment Task
© 2003-2005 WIDS-WTCSF
Online learning components were developed in WIDS.
Strategy 3: Create an online environment that
supports authentic assessment.
 Create a virtual environment that simulates an
authentic environment for performance.
 Use online communication tools such as threaded
discussion and chat to facilitate collaborative
planning, discussions, debates, role-plays and team
projects.
 Provide tools for the development of graphic
representations of complex concepts and processes
such as flowcharts and concept maps.
 Establish an effective vehicle for
submitting/presenting/documenting the product or
performance .
© 2003-2005 WIDS-WTCSF
Online Environment
 Online discussion group: debate/roll play
 Online tools that facilitate assessment:
groups, chat, group work spaces, graphic
capabilities
 Vehicles for documentation: discussion
board, web pages, electronic files, graphic
representations
© 2003-2005 WIDS-WTCSF
Regulatory Compliance Assessment
1.
2.
3.
4.
© 2003-2005 WIDS-WTCSF
Learners complete a Workplace
Assessment Checklist for a sample
manufacturing company based on case
study information located in course
materials.
Case Study information includes company
profile, regulatory documents, and sample
regulatory audit reports.
Learners work in groups to analyze
assessment results and make
recommendations.
Groups write a Recommendation
Summary for case study company.
Regulatory Compliance Assessment
© 2003-2005 WIDS-WTCSF
Online learning components were developed in WIDS.
Regulatory Compliance Assessment
© 2003-2005 WIDS-WTCSF
Online learning components were developed in WIDS.
Regulatory Compliance Assessment
© 2003-2005 WIDS-WTCSF
Online learning components were developed in WIDS.
Regulatory Compliance Assessment
© 2003-2005 WIDS-WTCSF
Online learning components were developed in WIDS.
Regulatory Compliance Assessment
© 2003-2005 WIDS-WTCSF
Online learning components were developed in WIDS.
Strategy 4: Provide opportunities for
practice and feedback.
 Incorporate into the learning plan
opportunities for practice and feedback.
 Base feedback on criteria detailed in
the scoring guide and provided to
learners.
 Use groups/peers to provide feedback
on practice performances/products.
© 2003-2005 WIDS-WTCSF
Practice and Learning
Metacognition
Learners need
feedback in order
to take
responsibility for
monitoring and
adjusting their
own learning.
© 2003-2005 WIDS-WTCSF
. . . hmm, the answer
to this question isn’t
what I thought it was.
What do I need to do
about it?
Post products in discussion forum for peer
feedback.
© 2003-2005 WIDS-WTCSF
Online learning components were developed in WIDS.
What about Question & Answer Assessments?
How do you assess and help learners test drive
their knowledge with higher level thinking skills?
Continually present questions that
require them to:
 make a decision
 draw a conclusion
 solve a problem
© 2003-2005 WIDS-WTCSF
What about Question & Answer Assessments?
Plan for performance; don’t just let them happen!
 Start with your competencies and learning




© 2003-2005 WIDS-WTCSF
objectives
Write question stems
Write correct answers
Write distractors
Edit
What about Question & Answer Assessments?
Strategies for Using Q&A Testing as Learning
 Present learners with questions that require
higher level thinking to make decisions,
determine solutions, or draw conclusions.
 Design questions so that learners apply
knowledge in the context of work or other life
roles.
 Provide corrective feedback for incorrect
answers.
© 2003-2005 WIDS-WTCSF
What about Question & Answer Assessments?
Strategies for Using Testing as Learning
 Build in branching to present additional
questions after missed questions.
 Teach learners early and often how to succeed
on tests and quizzes.
 Require learners to “test drive” their knowledge
by completing low stakes quizzes at all stages
of learning. [Could be self-check.]
© 2003-2005 WIDS-WTCSF
Practicalities
How authentic
can you be?
© 2003-2005 WIDS-WTCSF
How authentic can you be?
That depends on:






Target outcome (competency)
Placement in learning sequence
Access to online learning tools
Time constraints
Complexity of logistics
Control of environment and performance
parameters
 Number of students
© 2003-2005 WIDS-WTCSF
Performance Assessment and the
Learning Cycle
Application
Practice
© 2003-2005 WIDS-WTCSF
Motivation
Comprehension
Performance Assessment and the
Learning Cycle
Application
Practice
© 2003-2005 WIDS-WTCSF
Motivation
Comprehension
Online Performance Assessment Checklist
Assessment effectively evaluates the
performance of the target learning
outcome(s)
Assessment reflects a level of authenticity
that is effective and practical
Assessment can be accomplished online OR
in an environment accessible to the learner
Assessment includes criteria for performance
that are defined and available to learners as
a scoring guide
© 2003-2005 WIDS-WTCSF
Online Performance Assessment Checklist
 Assessment documents performance for
evaluation and feedback
 Assessment results in an artifact that will be
of use to the learner in the future
 Assessment makes effective use of
tools/resources available online for accessing
and sharing information
© 2003-2005 WIDS-WTCSF
Tools
Online Learning Assessment Planner
© 2003-2005 WIDS-WTCSF
Tools
Scenario-Type Question Worksheet
© 2003-2005 WIDS-WTCSF
Want to learn more?
© 2003-2005 WIDS-WTCSF
Performance Assessment in
Online Learning
What questions do
you have?
What bright ideas or
experience would
you like to share?
© 2003-2005 WIDS-WTCSF