MAINTAINING QUALITY IN BLENDED LEARNING: FROM
CLASSROOM ASSESSMENT TO IMPACT EVALUATION
PART I: DESIGNING AND EVALUATING LEARNING
Suzanne Weinstein, Ph.D.
Director of Instructional Consulting, Assessment, and Research
The Schreyer Institute for Teaching Excellence
The Pennsylvania State University
Part I Session Objectives
As a result of this workshop, participants will be able to…
 …write a student-centered, measureable learning
objective
 …map learning objectives to appropriate assessments
 …use course-level evaluation data for program
assessment
But first, some definitions
Assessment
“Assessment is the ongoing process of establishing
 clear, measurable expected outcomes of student
learning,
Suskie, L. (2009) Assessing student learning: A common sense guide.
San Francisco: Jossey-Bass.
Assessment
“Assessment is the ongoing process of establishing
 clear, measurable expected outcomes of student
learning,
 ensuring that students have sufficient opportunities to
achieve those outcomes,
Suskie, L. (2009) Assessing student learning: A common sense guide.
San Francisco: Jossey-Bass.
Assessment
“Assessment is the ongoing process of establishing
 clear, measurable expected outcomes of student
learning,
 ensuring that students have sufficient opportunities to
achieve those outcomes,
 systematically gathering, analyzing, and interpreting
evidence to determine how well students learning
matches the expectations and
Suskie, L. (2009) Assessing student learning: A common sense guide.
San Francisco: Jossey-Bass.
Assessment
“Assessment is the ongoing process of establishing
 clear, measurable expected outcomes of student learning,
 ensuring that students have sufficient opportunities to achieve
those outcomes,
 systematically gathering, analyzing, and interpreting evidence
to determine how well students learning matches the
expectations and
 using the resulting information to understand and IMPROVE
student learning.”
Suskie, L. (2009) Assessing student learning: A common sense guide. San
Francisco: Jossey-Bass.
Evaluation
“Evaluation is the analysis and use of data by faculty to
make judgments about student performance. Evaluation
includes the determination of a grade or a decision
regarding pass/fail for an individual assignment or for a
course.”
Goldman and Zakel (2009)
A blended or hybrid course is one in which
some face-to-face class periods are replaced
with online activities
+
Designing learning in a blended course
Designing learning in a (blended) course
1.
Learning
Objective
2.
Assignment
3.
Learning
Activities
Wiggins and McTighe (1998), Understanding by Design http://pixel.fhda.edu/id/six_facets.html
Designing learning in a (blended) course
Face-to -face
or
Online?
1.
Learning
Objective
2.
Assignment
3.
Learning
Activities
Designing learning in a (blended) course
Face-to -face
or
Online?
1.
Learning
Objective
2.
Assignment
3.
Learning
Activities
Integrate
face-to-face
and online
elements
Designing learning in a (blended) course
Learning
Objective
At the end of this
course, students
will be able to
evaluate an
empirical
research article
Assignment
Team project:
Find an empirical
article on a topic
of personal
interest and
collaboratively
evaluate the
article using a wiki
Learning
Activities
a. Students will
read an article
b. Instructor will
model an article
critique in class
http://pixel.fhda.edu/id/six_facets.html
Learning Objective
Taxonomy
Evaluation
Write a specific,
Match the objective to
learner-centered,
the closest equivalent
measureable objective of Bloom’s taxonomy
How you will evaluate
the students on this
objective?
What is the best format
for the evaluation?
At the end of this
Evaluating
course, students will
be able to evaluate
an empirical
research article.
Team project: Find
an empirical article
on a topic of
interest and
collaborate to
evaluate the article
using a wiki.
Teaching Strategies Face-to-face/Online
What teaching activity connection
you will use to prepare
students to meet
objective?
What is the best format
for the teaching
strategies?
How are the face-toface activities
connected to the online
activities?
Instructor will
model an article
critique (in class).
Instructor modeling
the critique links to
students practicing
the critique in the
online environment.
Adapted from Synthesis Worksheet: http://www.schreyerinstitute.psu.edu/pdf/Synthesis_Worksheet.pdf
Creating Learning Objectives
Identify what the learner should know or
be able to do by the end of the course
A learning objective should…
 …focus on the learner
 …be observable (i.e. measurable)
http://www.schreyerinstitute.psu.edu/Tools/LearningObj/
Use Bloom’s taxonomy (1956) to develop
learning objectives at various levels of
learning
Evaluation
Synthesis
Analysis
Application
Comprehension
Knowledge
Adapted from Eileen Herteis, The Gwenna Moss
Teaching & Learning Centre
If you prefer, you can also use Krathwohl
and Anderson version of the taxonomy
Creating
Evaluating
Analyzing
Applying
Understanding
Remembering
Adapted from Eileen Herteis, The Gwenna Moss
Teaching & Learning Centre
Chemistry Learning Objectives
Students will be able to:
 Identify the chemical deficiencies of the soil for a
crop.
 Distinguish between different types of chemicals
based on the chemical deficiencies of the soil for a
crop.
 Develop a solution for the soil deficiencies in a
proposal geared towards a business audience.
Evaluate the following learning objective
1. In this course you will understand the 3 major
communication disorders.
Evaluate the following learning objective
2. In this course we will cover the religion of ancient
Egypt.
Mapping Objectives to Assessment
Strategies (mere examples)
Remember
Understand
Apply
Traditional
Electronic
Multiple choice or
matching questions
Online quiz or
fact-based game
Multiple choice or
Matching questions
Multiple
choice
questions
Essay
summary
Online
quiz or
fact-based
game
Calculate Demonstrate Online
the answer a procedure
quiz
to a
problem
Flow chart
Concept
map
Do an
online
search
Make a
podcast
that
examines a
theory
http://edorigami.wikispaces.com/Bloom%27s+-+Introduction
Mapping Objectives to Assessment
Strategies (mere examples)
Traditional
Analyze
Distinguish
between 2
theories
Evaluate
Evaluate an
writing
sample
Create
Generate an
expository
essay
Analyze
data
Electronic
Reverse
engineer a
device
Conduct Collect and
and report
analyze
on a web media clips
survey
Appraise a
Judge a
Moderate Evaluate a
research
marketing
a
video or
article
plan
discussion
debate
forum
Formulate Produce and Generate
Create a
a research
deliver a
an evideo
plan
presentation Portfolio
Contrast 2
blog posts
Peer
evaluation
Build a
website
http://edorigami.wikispaces.com/Bloom%27s+-+Introduction
One last, but not least, important
consideration: Rubrics
Rubrics for digital assignments
http://edorigami.wikispaces.com/Rubrics+-+Bloom%27s+Digital+Taxonomy
Electronic assessment examples
History course objective
Students will be able to create personal histories
that link to specific historical times.
Family e-Album
Electronic assessment examples
Information Science and Technology
Students will be able to collaboratively develop
summaries of research related to online games
and virtual worlds.
IST 440 Research Project
Give it a try
Go to the following link. It will take you to a google
form. The questions will walk you through the
process of creating the appropriate assignment for
a particular objective.
https://spreadsheets.google.com/viewform?formkey
=dDZSbS1XOHI4ZngtMzZLNno4ZkVnWHc6MQ
Let’s see the results

Blended Learning Workshop Activity Spreadsheet
Use course-level assignments to measure
program goals
Psychology Program Goal
Students will be able to understand and interpret
psychological data, graphs and statistics.
Course Objective
Students will be able to correctly interpret a correlation
coefficient.
Performance by student
Exam 4
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
1
2
3
4
5
6
7
8
Percentage of students achieving
greater than 80%
Performance by learning objective
Exam 4 (Unit11-Unit12)
100%
80%
60%
40%
20%
0%
11.3;11.1
11.8
11.8
12.3
Learning Objectives
12.4
12.5;12.1
Resources

Hybrid Courses, University of Wisconsin at Milwaukee:
http://www4.uwm.edu/ltc/hybrid/

Schreyer Institute for Teaching Excellence, Penn State:
http://www.schreyerinstitute.psu.edu/

Bloom’s Digital Taxonomy:
http://edorigami.wikispaces.com/Bloom%27s+-+Introduction

Bloom’s Digital Taxonomy- Rubrics
http://edorigami.wikispaces.com/Rubrics+-Bloom%27s+Digital+Taxonomy

Evaluating Media Characteristics
http://www.elearnspace.org/Articles/mediacharacteristics.htm
Suzanne Weinstein
sweinstein@psu.edu
814-863-0689
http://www.schreyerinstitute.psu.edu
http://www.assess.psu.edu
CONTACT INFORMATION