Quality MattersTM
Overview
Updated 02112013
©MarylandOnline, Inc. 2013. All rights reserved
Agenda
• Program Introduction
• QM Basics
– 2011-2013 Quality Matters Higher Education Rubric
– Course Reviews and Certification
– Professional Development and Certification
• What’s In It for Faculty
• What’s In It for Institutions
• The QM Organization
©MarylandOnline, Inc. 2013. All rights reserved
The Quality Matters Program
Quality Assurance through Faculty
Development and Course Design
©MarylandOnline, Inc. 2013. All rights reserved
Academic & Non-Profit
• Quality Matters is a not-for-profit subscription
service providing tools and training for quality
assurance of online courses
• Developed by MarylandOnline with funding from
FIPSE, it was designed by faculty for faculty and
focused on improving student learning
• Adopted by a large and broad user base, QM
represents a shared understanding of quality in
online course design
©MarylandOnline, Inc. 2013. All rights reserved
Quality Matters is …….
• A set of standards (Rubric) for the design of online
and blended courses
• A peer review process (faculty to faculty) for
reviewing and improving online and hybrid courses
• A faculty support tool used by instructional
development staff
• A professional development opportunity
©MarylandOnline, Inc. 2013. All rights reserved
The QM Approach
QM is a faculty-driven, peer review process that is…
Collaborative
Collegial
Continuous
Centered
-
in academic foundation
around student learning
©MarylandOnline, Inc. 2013. All rights reserved
Factors Affecting Course Quality
QM ONLY Reviews Course Design
©MarylandOnline, Inc. 2013. All rights reserved
QM’s Role in Quality Assurance
• QM looks at course design
•
The harnessing of technology to deliver instruction and promote student learning
• QM provides a process for peer-to-peer feedback for
faculty in the continuous improvement of their
course and certifies courses as meeting shared
standards of best practice
• Quality Matters is not the complete answer to
quality assurance for online education, but it can be
a critical component
©MarylandOnline, Inc. 2011. All rights reserved
What QM is NOT About…
• Not about an individual instructor
(it’s about the course)
• Not about faculty evaluation
(it’s about course quality)
• Not about judgment
(it’s about diagnosis and improvement)
• Not about “win/lose” or “pass/fail”
(it’s about continuous improvement in a supportive
environment)
©MarylandOnline, Inc. 2011. All rights reserved
QM as a National Standard
• 600+ current subscribers
• 46 states represented
• QM has trained 22,000+ faculty and
instructional design staff
• Community Collaboration
• Largest community collaboration on
quality in online education
• Award Winning
• - Sloan-C , USDLA, WCET
©MarylandOnline, Inc. 2013. All rights reserved
Quality Matters Subscribers
WASHINGTON
NORTH
DAKOTA
MONTANA
MAINE
MINNESOTA
OREGON
VT
NH
WISCONSIN
SOUTH
DAKOTA
IDAHO
NEBRASKA
NEVADA
NEW YORK
MICHIGAN
WYOMING
RI
CONN
PENN
IOWA
OHIO
UTAH
ILLINOIS
COLORADO
WV
MARYLAND
VIRGINIA
MISSOURI
KENTUCKY
CALIFORNIA
NORTH
CAROLINA
TENNESSEE
OKLAHOMA
NEW
JERSEY
DELAWARE
INDIANA
KANSAS
ARIZONA
MASS
ARKANSAS
SOUTH
CAROLINA
NEW MEXICO
MISS
ALABAMA
GEORGIA
TEXAS
FLORIDA
LOUISIANA
PUERTO RICO
Current Subscribers
Consortium Subscribers
ALASKA
Statewide Subscribers
HAWAII
©MarylandOnline, Inc. 2013. All rights
reserved
VIRGIN
ISLANDS
QM Basics
More than the sum of its three parts
©MarylandOnline, Inc. 2011. All rights reserved
The Rubric is the Core of QM
Consisting of:
• 8 key areas (general standards) of course quality
• 41 specific review standards
• 21 essential standards
• Detailed annotations and examples of good practice
for all 41 standards
©MarylandOnline, Inc. 2013. All rights reserved
The QM Rubric
Eight General Standards:
1.
2.
3.
4.
5.
6.
7.
8.
Course Overview and Introduction
Learning Objectives (Competencies)
Assessment and Measurement
Instructional Materials
Learner Interaction & Engagement
Course Technology
Learner Support
Accessibility
Key components must align.
Alignment: Critical course elements work together to ensure
that students achieve the desired learning outcomes.
©MarylandOnline, Inc. 2013. All rights reserved
Standards Supported by Research
Course Reviews and Certification
The Peer Review Process
©MarylandOnline, Inc. 2013. All rights reserved
The QM Peer Review Process
©MarylandOnline, Inc. 2013. All rights reserved
Types of Course Reviews
• Unofficial Reviews
⁻ Internal/Informal subscriber reviews not required
to follow QM process
• Official Reviews – follows QM policies/protocols
⁻ QM-Managed review - QM manages and pays
review team
⁻ Subscriber-Managed review - Subscriber manages
and pays review team
©MarylandOnline, Inc. 2013. All rights reserved
QM = Process and Rubric
Process
OFFICIAL
INFORMAL
Rubric
Outcome: Earn QM recognition
Must follow official QM guidelines
and procedures
Outcome: Earn QM recognition
Tool to assess online courses during
formal QM review
Outcome: Improve courses, meet
institutional goals, demonstrate
commitment to quality
Outcome: Improve courses,
meet institutional goals
Guide to develop new online
courses AND review and update
online courses
Outcome: Improve courses, meet
institutional goals, demonstrate
commitment to quality
Institutions determine use and
procedures
Outcome: Improve courses, meet
institutional goals, demonstrate
commitment to quality
Guide to develop new online
courses AND review and update
online courses
The Peer Review Team
On an official review, Review Teams consist of:
• Three QM-Certified Peer Reviewers
• Pre-requisite: Current online teaching experience & training
• Master Reviewer as Team Chair
• Peer Reviewer with additional experience and training
• One reviewer must be a subject matter expert
• At least one reviewer must be external to the institution
sponsoring the course
AND
• Faculty course developer:
• access to rubric prior to review
• involved in pre-review discussions
• consulted during review
Peer Feedback
The peer feedback provided in a course
review is collegial in tone and language;
however, the reviews are rigorous and even
courses that meet standards will benefit
from the detailed, specific, and relevant
feedback provided.
©MarylandOnline, Inc. 2013. All rights reserved
QM Certification of Courses
• Course reviews take 4-6 weeks; 20-week
window for reviews, including any
amendments, to be completed
• QM recognition is provided
by year recognized
• QM logo on course and catalog; registry
on QM Web site
©MarylandOnline, Inc. 2013. All rights reserved
Professional Development and
Certification
QM Workshops & Certifications
©MarylandOnline, Inc. 2013. All rights reserved
QM Workshops & Certifications
Delivery Options:
•
Online – virtual, asynchronous
•
Onsite – face-to-face, synchronous
•
Collaborate – virtual, synchronous
Focus:
•
QM Implementation
•
Prepare faculty and staff to hold QM roles
•
Faculty Development Workshops
•
Prepare faculty to design and improve
courses
©MarylandOnline, Inc. 2013. All rights reserved
Faculty Development Workshops
Course-focused Workshops
•
•
•
•
Applying the QM Rubric*
Designing Your Online Course
Improving Your Online Course
Designing Your Blended Course
*Note: Flagship training is also
required for QM Implementation
Standards-Focused Workshops
• Design that Welcomes Your
Students
• Creating a Foundation with
Learning Objectives
• Choosing and Using Media
Effectively
• Addressing Accessibility
Alignment Standards Workshops
• Connecting Learning Objectives
and Assessments
• Linking Instructional Materials
and Learner Engagement
©MarylandOnline, Inc. 2013. All rights reserved
QM Implementation: Certifications
•
•
•
•
Applying the QM Rubric (APPQMR)
Peer Reviewer Course*
Master Reviewer Certification*
Certified Facilitator*
• F2F CF: For face-to-face delivery of APPQMR
• COF: For online delivery of APPQMR
• Institution Representative Training*
*Offered online only
©MarylandOnline, Inc. 2013. All rights reserved
Capacity Building: QM Roles & Criteria
Peer Reviewers
Independent
Trainers
Master
Reviewers
Institution
Representative
Current Online
Instructor
Applying the QM
Rubric Workshop
Certified QM Peer
Reviewer
Applying the QM
Rubric Workshop
Applying the QM
Rubric Workshop
(online or F2F)
Peer Reviewer
Course
(certification not
required)
Experience on 2+
course reviews
Institution
Representative
Course
Peer Reviewer
Course (online)
Certified Trainer
Workshop
Master Reviewer
Workshop
Institution may
designate
additional IRs
Submission of
application &
MOU (for
certification)
Institution on Full
or Statewide
Subscription
Annual Recertification
©MarylandOnline, Inc. 2013. All rights reserved
Quality Matters:
By Faculty for Faculty
What’s In It for Faculty
©MarylandOnline, Inc. 2013. All rights reserved
What’s In It For Faculty?
• Improvement of online courses
• External quality assurance
• Participation useful for professional
development plan and portfolio
• Faculty development trainings
• Review other courses and gain new ideas for
own course; expand professional community
Roles for Faculty in QM
Peer
Reviewers
Master
Reviewers
Applying the
QM Rubric
Workshop
Certified QM
Peer Reviewer
Peer Reviewer
Certification
(online)
Experience on
2+ course
reviews
Current online
instructor
Master
Reviewer
Certification
Tools in the Spirit of the Academy
• Standards grounded in the research
literature and best practices
• Tools and process developed by
faculty and instructional design
experts for faculty support
• Peer review and other opportunities
to collaborate and mentor around
quality in online learning
What to Expect from Course Reviews
• Consistency and Rigor
• Professionalism and Commitment
• Useful and Constructive Feedback
• Positive Outcomes for Faculty
• Less than 50% of courses in QM-managed
reviews meet standards upon initial review (but
all can meet after amendment)
Review Exit Survey Comments
Sample Comments from Faculty Developers
• “I was too close to see what could be improved.”
• “Provides a great way to get an objective view of
your course.”
• “It made all of my online courses better.”
• “It provides a view from a more student oriented
perspective.”
• “Many elements that might contribute to a student
withdrawing can be eliminated.”
Review Exit Survey Comments
Sample Comments from Peer Reviewers
• “I always learn new wonderful approaches from
viewing other courses and working with expert peer
reviewers.”
• “Whenever I read the annotations, I make changes
to my courses.”
• “I always find reviews to be a learning experience.”
• “This experience was very rewarding for me as both
an educator and course developer.”
• “It’s a great tool for honing your own course.”
Research Opportunities and Support
• QM encourages research through
– Consultation with QM Director of Research
– Collaboration on research and data resources
– Participation in faculty surveys of all QM
subscribers
– Sponsorship of research - see RFP at
www.qmprogram.org
Quality Matters:
A Multi-faceted Resource
What’s In It for the Institution
©MarylandOnline, Inc. 2013. All rights reserved
Multiple Uses of QM
Reported Uses of QM Tools:
• Guidelines for initial online course development
• Quality assurance of existing courses
• Ongoing faculty professional development
• Institutional re-accreditation packages
• Focus attention on distance learning policies &
steering committees
• Create a campus climate and structure that
promotes teaching and learning
QM Professional Development
Reported uses:
•
Educate faculty about quality in online courses
•
Reduce time and labor in designing courses.
•
Provide professional development opportunities
for faculty and staff
•
Threshold requirement for adjunct faculty
•
Inform teaching practices related to design and
structure for BOTH online and F2F courses
•
Quality process indicators for accreditation selfstudies
©MarylandOnline, Inc. 2013. All rights reserved
Subscription Benefits
•
•
•
•
•
•
•
•
Current QM Rubric and related rubrics
Fully automated QM Rubric Tool
Subscriber Discounts
IR Training
MyQM & User’s Group
Internal Course Reviews
Training License*
Subscriber-Managed Course Reviews*
* Depends on subscription type
©MarylandOnline, Inc. 2013. All rights
reserved
Quality Matters:
A Collaboration on Quality
The QM Community
©MarylandOnline, Inc. 2013. All rights reserved
The QM Community
Online Learning Students
QM Institutions:
HE, K-12, Publishers,
Partners, Sponsors
Committees: Academic Advisory, Rubric,
Training, Reviews
IRs, Trainers, Peer
Reviewers
QM Staff
& Consultants
©MarylandOnline, Inc. 2013. All rights reserved
QM is Continuous Improvement
• Improved courses
• Engaged faculty
• Reduced course
development time
• Ongoing faculty
development
• Quality benchmarking
• Awareness & support
for online learning
• Institutional
improvement
©MarylandOnline, Inc. 2013. All rights
reserved
Your Questions?
Please contact the appropriate QM
representative – available to you via
email, Skype or phone.
©MarylandOnline, Inc. 2013. All rights reserved
We are here for you! QM Contacts:
• Deb Adair,
Managing Director
dadair@qualitymatters.org
• Grace Hall,
Communications Manager
ghall@qualitymatters.org
• Nancy Ragias,
QM Registrar, Higher Education
nragias@qualitymatters.org
• Marijane Hakun,
Subscriber Services Manager
mhakun@qualitymatters.org
• Melissa Poole,
Course Review Manager
mpoole@qualitymatters.org
• Julie Henn,
CIO; Director, Technology
jhenn@qualitymatters.org
©MarylandOnline, Inc. 2013. All rights reserved