Alternative and Performance-based
Assessment in Online Learning: Analysis,
Framework, and Strategies to Systematically Assess
Student Learning
Mahnaz Moallem, Ph.D.
The University of North Carolina at
Wilmington
AECT 2004 Conference
Oct. 10–22, Chicago, IL
Purposes of Presentation
{
{
{
{
{
Analyze Student-centered, Project-based,
Performance-based Assessment system
(SPPA) to identify its implementation
problems in face-to-face courses
Identify potential of online learning
environment for implementation of SPPA
system
Propose a design framework for SPPA
system in online courses
Demonstrate the application of the model
using an example online course
Share some of the enablers and barriers
A Conventional Teacher-Centered &
Outcome-based Assessment Model
Outcome
Establish
Learning
Outcomes
Instruction
Identify &
Develop
Assessment
Implement
Instruction
Assessment
Assess
Learning
Outcomes &
Effectiveness of
Instruction
Re-mediate and/or Revise Instruction
Student-centered, Project-based,
Performance-based Assessment system
lly
g
a
c
n
ri enti
u
s
ea auth
m
rts ore
o
pp n g m
u
S
ni
r
a
le
En
a
cri bles
t
gre ical stud
ate thin ent
s
k
rd
ep ing s to d
i
th
of kills splay
kn
a
ow nd
led
ge
De
v
ove elops
r th
ad
e
Fos
c ou e e p e
rse r di
g ro t er b o
ma alog
u
t
h
p
ter
act
i
o
n
r
ials
d
ien
ivit
i
v
i
t
d
ies
ed
lea ual an
rni
ng d
Provides a diversity of
learning opportunities
And most importantly………
Student-centered, Project-based,
Performance-based Assessment system
Emphasizes:
{ Integrating
assessment &
instruction
{ Progress in learning
& in becoming more
sophisticated
learners
{ Pedagogical changes
in teaching methods
Student-centered, Project-based, Performance-based
Assessment system
Self-Assessment/
Feedback
Expert-Assessment/
Feedback
Performance-based,
Process-based,
Progress-based
Assessment
Peer-Assessment/
Feedback
Continuous Assessment
Continuous Assessment
Continuous Assessment
Real World Problem or Project
SPPA System in Face-to-Face
Classrooms: Implementation Problems
•
While the concept
of dynamic &
continuous
assessment is
appealing it is
difficult to manage
timely and
formative feedback
SPPA System in Face-to-Face
Classrooms: Implementation Problems
{
Limitation of time,
tools & resources for
gathering &
documenting dayto-day assessment
data constrains the
effectiveness of
SPPA system
SPPA System in Face-to-Face
Classrooms: Implementation Problems
It is difficult to:
{ Be an effective
facilitator
{ Ask appropriate
guiding questions
{ Document student
progress when
solving
problems/projects
SPPA System in Face-to-Face
Classrooms: Implementation Problems
It is difficult to:
• Promote individual
students’ growth
without providing
individual assessment
• Keep the focus of
individual
assignments on selfassessment
SPPA System in Face-to-Face
Classrooms: Implementation Problems
•
•
Self-assessment tools
do not seem to work
effectively without
the instructor’s
feedback & guidance.
It is difficult to
document the effects
of self-assessment
tools on individual
growth & team
performance
SPPA System in Face-to-Face
Classrooms: Implementation Problems
•
Given limitation of
time and tools,
using multiple
sources of
assessment is
difficult to manage
SPPA System in Online learning
Environment
Online learning environment:
{ Has the highest potential for real
world, authentic performance
assessment
{ Offers new tools and resources that
make implementation of SPPA a
reality
{ Provides a much better environment
for practicing duties of a mentor,
coach and/or facilitator
Online Learning Environment: New
Tools & Resources
•
•
Database and
interactive system
to track, monitor &
document students’
activities
automatically
Easy access & easy
development
process for
conventional
assessment tools
Online Learning Environment: New
Tools & Resources
{
{
Opportunity to selfassess own
knowledge through
automatic and
instant feedback
Multiple
communication tools
to facilitate as well
as to document
dialogue
Online Learning Environment: New
Tools & Resources
Content & communication
tools to:
{ Develop projects/tasks
that could incorporate
multiple resources &
perspectives
{ Complete project/tasks
at various intervals
{ Provide continuous
formal & informal
feedback
Online Learning Environment: New
Tools & Resources
{
{
{
Unlimited & self-paced
access to course materials
& communication tools
Interactive & asynchronous
communication tools to
promote thoughtful &
reflective commentaries
Learner’s & instructor’s
access to the record of
responses, answers,
feedback, etc. to analyze
and interpret one’s own
performance & to reflect
A Model for Assessment of Online
Learning
Real World Project or Problem
ExpertAssessment
SelfAssessment
ExpertAssessment
SelfAssessment
ExpertAssessment
SelfAssessment
Initial
Assessment
Progress
Assessment
Product
Assessment
PeerAssessment
PeerAssessment
PeerAssessment
Designing Online SPPA System:
An Example
Product Assessment: Designing Real
World Problem/Project
{
{
{
{
The first step was to develop a
complex and multifaceted real world
problem or project that required
students to
Demonstrate achievement of all
course goals
Engage in challenging real world
activities
Work in teams
Work with real clients and/or the
professional community
Product Assessment: Designing Real
World Problem/Project
{
{
{
{
{
Made sure that the real world
problem/project was complex enough to:
Break it into several subtasks or mini
projects to be completed at various
intervals
Monitor its subtasks
Structure the accomplishment of the
major task
Develop a timeline for student work
Incorporate process-based, continuous
assessment & feedback
Product Assessment: Designing Self
Assessment Tools
{
{
{
{
A series of self-assessment rubrics was
developed (self assessment)
One or two completed projects were linked
to the course project to help set the
standards for best performance (selfassessment)
The instructor reviewed teams’ progress
and product and provided written feedback
(expert assessment)
The instructor scheduled a chat session
with each team and reviewed the written
comments at least once for each part of
the project (expert assessment)
Product Assessment: Designing Self
Assessment Tools
{
{
Provided an online space for students
to work on their course project
continuously, and in collaboration with
their team members throughout the
semester (peer assessment)
Used “Student Presentations” area
within WebCT course management
system (peer & expert assessment)
Assessing Progress
{
{
{
{
The next step in designing SPPA system
was to develop related problems or cases
for each unit of instruction to:
Provide learning experiences for the
completion of the course project
Address domain specific knowledge and
skills to complete the course project
Identify & further developed resources
and supporting instructional materials
Develop and form unit content and
instruction
Assessing Progress: Expert, Peer &
Self- Assessment Tools
{
{
{
{
{
Self-assessment quizzes to assess
background knowledge with automatic
expert feedback (self & expert assessment)
Individual assignments and forum
discussion to assess understanding and
synthesis of readings and domain specific
knowledge with immediate expert and peer
feedback (Peer & Expert assessment)
Conversation space for teams working on
unit problems (peer & expert assessment)
Team evaluation instruments to assess
team’s performance (peer assessment)
Opportunity to review and compare other
teams’ products and feedback (peer
assessment)
Initial Assessment
{
{
{
The third step was to develop opportunities
for assessing students’ initial
understanding of the materials in order to
assist:
Students in making their initial thoughts
explicit to themselves
Students compare their initial
understanding with their later
conceptualization of the materials &
identify progress
The instructor to identify coaching and
guiding strategies
Initial Assessment: Self, Peer & Expert
Assessment Tools
{
{
{
{
Individual assignments that consisted
of a list of thinking questions
Non-graded, self-assessment quizzes
with immediate feedback using WebCT
Quiz Maker
Team Forum discussion
Learning, thinking & multiple
intelligences inventories to identify
one’s own learning and thinking styles
Reflections: Enablers
{
{
{
Use of SPPA Systems made assessment a
natural extension of instruction
Non-graded and reusable instructional
activities gave both students and the
instructor useful feedback on teachinglearning process
Constant feedback enabled students to be
strategic in their own learning process and
enabled the instructor to adapt the
instructional process to meet the needs of
students
Reflections: Enablers
{
{
{
{
Assessment of actual performance enabled
students to translate what they had
learned in different units of instruction to a
real-life application
Within SPPA system it was possible to
provide a wide range of assessment tools
and techniques
SPPA offered more opportunities and ways
for providing feedback
SPPA made the management, collection
and transfer of assessment information
easier
Reflections: Barriers
{
{
{
Accessibility and responsiveness of
the instructor
Instructor’s philosophy of learning
and assessment and his/her
expectations of students
Fostering a participatory online
learning environment