Quality Matters: A FIPSE project
for Peer Course Review
A Grant Initiative of MarylandOnline
Sponsored by the
U.S. Department of Education
Fund for the Improvement of Secondary Education (FIPSE)
http://www.qualitymatters.org/
1
Mary Wells
(Co-Director)
Prince George’s Community College, MD
WellsML@pgcc.edu
Dr. Chris Sax
(Co-Director)
University of Maryland, University College
csax@umuc.edu
Dr. Joan McMahon
(Process Committee Co-Chair)
Towson University, MD
mcmahon@towson.edu
http://www.qualitymatters.org/
2
FIPSE is competitive
• 1550 preliminary applications
• 150 invited to submit proposal
• 42 funded projects (<3 %)
Quality Matters!
http://www.qualitymatters.org/
3
Inter-Institutional Sharing
Maryland Online (MOL) is a statewide
consortium of 10 Maryland community colleges
and senior institutions that share courses.
http://www.qualitymatters.org/
4
Quality Matters: Key Outcomes
• Replicable pathway for inter-institutional quality
assurance (QA) of online courses
• Faculty-centered, peer review-based, consortiumwide QA process
• Review tools that incorporate nationally
recognized standards of best practice
• F2F and online training
• Expanded resource sharing and increased
articulation agreements
http://www.qualitymatters.org/
5
Quality Matters:
Inter-Institutional Quality
Assurance in Online Learning
FIPSE Award:
$509,177
Available:
September 2003
Term:
3 Years (9/03 -6/06)
Fiscal Agent:
Prince George’s Community
College (MD)
Web Site:
http://www.qualitymatters.org
http://www.qualitymatters.org/
6
What this process is NOT
• Not about an individual
instructor (it’s about the
course)
• Not about faculty evaluation
(it’s about course quality)
• Not about “winners” and
“losers” (it’s about continuous
improvement in a supportive
environment)
http://www.qualitymatters.org/
7
For Our Purposes, Quality Is…
• More than average;
more than “good
enough”
• An attempt to capture
what’s expected in an
effective online course
• Based on research and
widely accepted
standards
100
Higher
Standard
50
0
http://www.qualitymatters.org/
8
Quality Matters
How do we …
– identify &
recognize it?
http://www.qualitymatters.org/
• Determine the Criteria
of the Reviewers?
• Develop a Rubric on
Quality?
9
Session Objectives
• Develop a rubric by:
– Examining different brands of cookies.
– Determining the criteria for review.
– Setting the standards for the review.
• Reflect on the process of determining quality
issues and standards.
• Project the implications for the process in online
course review.
http://www.qualitymatters.org/
10
Develop a Rubric
1. Determine the criteria of
the reviewer
– You must have had some
experience in eating.
http://www.qualitymatters.org/
11
2. Examine the “goods.”
– Open the different bags
of chocolate chip
cookies.
http://www.qualitymatters.org/
12
3. Establish the
weighted criteria
of the “goods”.
These are your
standards.
http://www.qualitymatters.org/
• What are the criteria?
• What weight will you
give each criteria?
13
4. Rate and rank
the “goods.”
•
How did you test the
cookies for the
quality indicators?
•
http://www.qualitymatters.org/
How were your
criteria results
recorded?
14
5. Revise the criteria
• What criteria were the
main differentials? Or
the “deal breakers?”
• What insights did you
have about the criteria
development/rating
process?
http://www.qualitymatters.org/
15
The Reflection
1.
2.
3.
4.
5.
6.
How did your group achieve consensus on the ratings?
What were the greatest challenges to arriving at a
consensus ranking?
How comfortable would you be if you had to defend this
rating before your vice president or before the faculty
association?
Can your criteria be applied to all such similar items?
Is this process replicable? Can you use this process for
other quality assurance activities?
What advice would you give to other quality assurance
teams?
http://www.qualitymatters.org/
16
Cookies Don’t Just Happen!
•
•
•
•
•
•
Recipe
Ingredients
Processes
Packaging
Marketing
Seal of Approval
http://www.qualitymatters.org/
What’s similar to
quality online
courses?
17
Analogy
Cookies to Online Course Review
Issues
How long was the Time on Task?
Cookies
Online Courses
15 minutes
Who sets the criteria?
Who determines the weighting standards?
Is it replicable?
Are there subjective vs objective
indicators?
What are the “deal breakers?”
How many can be evaluated at each
sitting?
What are the experiences of the
reviewers?
What were the unifying elements?
http://www.qualitymatters.org/
18
Now What?
• What do we do with
“quality” courses?
• How do we use our
certified peer reviewers?
• How do we tell students
what this means?
http://www.qualitymatters.org/
19
Session Evaluation
For grant purposes, please complete
the evaluation provided.
And thank you for your interest in the
project.
http://www.qualitymatters.org/
20
Additional Information
Please provide your email address if
you wish to learn more.
http://www.qualitymatters.org/
21
Additional Slides…
http://www.qualitymatters.org/
22
Quality Matters:
QA Process Overview
• Review of online courses by teams of 3 faculty
members
• Using Rubric tied to national standards of best
practice
• Feedback provided for improvement of course
quality
• Instructional design support provided to
implement recommendations
• Certification of course when standards met
http://www.qualitymatters.org/
23
Goals for Selecting Courses
•
•
•
•
•
Focus on quality
Continuous improvement
Positive environment
Low-risk
Increase number of
“shareable” courses
http://www.qualitymatters.org/
24
Quality Matters: Courses
• MarylandOnline Shared Course List
• State of MD Critical Workforce Needs:
– Engineering
– Teacher Education
– Information Technology
– General Studies
– Allied Health
http://www.qualitymatters.org/
25
Quality Matters:
Project Management Team
Project Co-Directors
Mary Wells
Chris Sax
Prince George’s CC
UMUC
Project Management Team
Kay Kane
Cynthia France
Jurgen Hilke
Wendy Gilbert
John Sener
Project Coordinator
Chesapeake College
Frederick CC
MarylandOnline
Project Evaluator
http://www.qualitymatters.org/
26