Quality Assurance (QA) Rubric
Course Code:
Course Title:
Term Deployed:
Instructional Designer:
Associate Dean (AD):
Assistant Dean (AsD):
AQ Team Member
Date Reviewed (MM/DD/YYYY):
Summary [ex. Driving concerns, opportunities for improvement, biggest risks for student success, major recommendations]
Quick Links (QL)
Course Foundation
Course Outcomes
Cohesive Design
Learning Objectives
Introduction
Vehicle
☐
Score:
Assessments
Alignment
Formative Assessments
Authenticity
Formative Grading Metrics
Summative Grading Metrics
Weight
Exemplary (80-100)
☐
Satisfactory (60-79)
Instruction
Scaffolding
Content Accuracy
Workload
Modalities
Instructional Clarity
☐
Needs Improvement (40-59)
Student Experience
Organization
Accessibility
Automation
Tools and Technology
Polish
☐
Significant Issues (0-39)
Course Foundation
29
The course is founded on a cogent, relevant, and appropriate vision that drives the essential approach and design.
Critical
Elements
Exemplary
Course
Outcomes
Meets “Satisfactory” and course
outcomes visibly map to program
outcomes and embody framework
principles (ex. multi-dimensional,
enduring, relevant, and terminal)
QL
Learning
Objectives
☐
(8)
Meets “Satisfactory” and objectives
embody framework principles (ex. unidimensional, discrete, logical learning
path to outcomes)
QL
☐
(6)
Satisfactory
Outcomes are measurable and
represent appropriate scope,
complexity, and rigor for the program
and level
☐
Significant Issue
Pts.
Outcomes are immeasurable or do not Outcomes are not explicit, coherent,
represent appropriate scope,
or student-centered
complexity, or rigor for the program
and level
☐
(6)
Learning objectives are measurable
performances assessed in the module
and all map to at least one of the
course outcomes
☐
Area of Improvement
(4)
☐
(0)
Learning objectives are not
Learning objectives are not explicit,
measurable, do not represent
coherent or student-centered
performances assessed in the module,
or do not all map to at least one of the
course outcomes
(4.5)
This work is licensed under the Creative Commons Attribution-ShareAlike 3.0 Unported License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/3.0/deed.en_US.
☐
(3)
☐
8
6
(0)
Page 1 of 8
Vehicle
Meets “Satisfactory” and the contextual All activity vehicles are pedagogically Some activity vehicles are not
purpose of the vehicle is transparently appropriate for the type of student
pedagogically appropriate for the type
communicated
performance or experience desired of student performance or experience
desired
QL
☐
(6)
☐
Cohesive Design Meets “Satisfactory” and the course
Employs central themes, draws
narrative purposefully and meaningfully connections to previous topics, and
guides students along the learning path stages future learning to create
cohesion between modules
QL
Introduction
QL
☐
(5)
Meets “Satisfactory” and includes a
conceptual map or other creative
representation of the essential nature
and approach of the course
☐
(4)
☐
(4.5)
☐
☐
There are missed opportunities for
threading course themes, drawing
connections to previous topics, or
staging future learning
☐
(4)
Introduction situates the course
within the larger body of knowledge
and defends its relevance for
professional and personal contexts
(3)
(2.5)
Course introduction is a basic
description of course topics and
activities or lacks relevance to
professional or personal contexts
☐
(3)
(2)
Activity vehicles are systemically
inappropriate or could impair the
student’s ability to successfully
complete the assignment
☐
6
(0)
Course content is siloed in disparate
modules, giving the impression that
the component parts of the course are
not connected
☐
(0)
Introduction is erroneous, misaligned,
or not explicit
☐
29
The course contains assessments that effectively promote and measure student mastery of outcomes in a reliable and valid manner.
Exemplary
Alignment
Satisfactory
Meets “Satisfactory” and the alignment All key performances of the
is made explicit for students
summative assessments map back to
at least one course outcome and each
of the outcomes is summatively
assessed
QL
Authenticity
☐
(6)
☐
(4.5)
Summative assessments fully represent Components of the summative
or emulate real-world performances or assessments represent or emulate
applications
real-world performances or
applications
QL
☐
(5)
☐
(4)
Area of Improvement
Not all key performances of the
summative assessments map back to
at least one course outcome or the
outcomes are not sufficiently
summatively assessed
☐
(3)
Components of the summative
assessments do not represent or
emulate real-world performances or
applications
☐
(2.5)
4
(0)
Assessments
Critical
Elements
5
Significant Issue
Pts.
There is a significant misalignment
between the summative assessments
and the outcomes
6
☐
(0)
The summative assessments are
conceptually inauthentic given other
assessment options
☐
5
(0)
Page 2 of 8
Summative Meets “Satisfactory” and grading
Grading Metrics metrics include annotations (ex.
glossary, exemplars) to build clarity of
expectations and objectivity in scoring
QL
Formative
Assessments
QL
☐
(5)
Meets “Satisfactory” and formative
assessments represent true practice
opportunities for the summative
assessments
☐
(4)
Formative
Meets “Satisfactory” and grading
Grading Metrics metrics are reliable and low-inference
QL
Weight
☐
(4)
All summative grading metrics are
valid and reliable for the constructs
being measured
☐
(4)
Formative opportunities are lowstakes and directly align to
summative assessments
☐
(3)
All formative activities indicate a
grading metric and all grading metrics
are valid instructional tools that align
to the summative grading metrics
☐
(3)
Not all summative grading metrics are There are summative assessments
valid and reliable for the constructs
without associated grading metrics or
being measured
the grading metrics are systemically
not valid and reliable for the
constructs being measured
☐
There are opportunities for formative
assessments that are not taken or
those that are present are not lowstakes
☐
QL
☐
(5)
☐
(4)
☐
☐
☐
☐
27
The instructional materials of the course promote student engagement and meaningful learning.
Exemplary
Scaffolding
Meets “Satisfactory” and instructional
materials represent an innovative or
creative approach for creating student
engagement and success
QL
☐
(6)
Satisfactory
Area of Improvement
All instructional materials have direct Some instructional materials are
relevancy and set students up for
superfluous or only tangentially
success in the assessments
related to the assessments
☐
(4.5)
☐
(3)
5
(0)
Instruction
Critical
Elements
4
(0)
There is a significant break between
the method for determining the
course grade and the demonstration
of competency.
(2.5)
4
(0)
Some formative activities do not
indicate the appropriate grading
metric or the grading metrics are
systemically invalid instructional tools
or not aligned to the summative
grading metrics
(2)
Meets “Satisfactory” and grade weight Assessments are given appropriate
The weighting of assessments is
schema aligns with the programmatic weight to ensure that the summative potentially misaligned to student
vision or guidelines
assessments are critical for passing
mastery of the course outcomes
the course
(0)
There are many opportunities for
formative assessments that are not
taken or those present would
negatively impact student success
(2)
All formative activities indicate a
grading metric, but not all grading
metrics are valid instructional tools
that align to the summative grading
metrics
☐
☐
(2.5)
5
Significant Issue
Pts.
Instructional materials are
systemically irrelevant or do not set
students up for success in the
assessments
☐
6
(0)
Page 3 of 8
Workload
Meets “Satisfactory” and creative
methods for distributing workload
without inhibiting rigor are employed
QL
Instructional
Clarity
QL
Content
Accuracy
☐
(6)
Meets “Satisfactory” and directions
exemplify positive, economical, and
effective communication
☐
(6)
Meets “Satisfactory” and subjectmatter experts involved are considered
pillars in the field and bring added
credibility or value to the course
QL
Modalities
☐
(5)
Course activities, materials, and
workload reflect appropriate
difficulty and complexity for the level
of the course
☐
QL
☐
(4)
☐
☐
(3)
Content sources did not undergo
review to ensure accuracy or
contributors were not appropriately
qualified and credentialed
☐
(4)
Meets “Satisfactory” and innovative or Course activities and materials
creative modalities for active learning provide students with multiple
are employed
modalities for active learning
(3)
Course activities, materials, and
workload are systemically
inappropriate in difficulty and
complexity for the level of the course
☐
(2.5)
Modality or active learning
opportunities were missed
☐
(3)
(2)
☐
Student Experience
QL
Automation
Meets “Satisfactory” and employs a
highly intuitive and user-friendly
interface
☐
(3)
The appropriate evaluation of higherorder critical tasks is automated using
innovative or creative practices
QL
☐
(3)
Course navigation and usability
supports the learning process
☐
(2)
15
(1.5)
Wherever possible and appropriate, Opportunities for automated
the evaluation of assessments is done assessments were present but not
in an automated fashion
utilized in the course
☐
(2)
Pts.
There are minor issues with the course Course navigation and usability
navigation and usability
interfere with the learning process
☐
☐
(1.5)
☐
3
(0)
The course does not meet
requirements for automation set by
the program director or the use of
automation negatively impacts
student success
☐
4
(0)
Technology, required materials, and user interface facilitate student success and reflect the professionalism and polish of COCE and the university.
Critical
Exemplary
Satisfactory
Area of Improvement
Significant Issue
Elements
Organization
5
(0)
The lack of modalities or active
learning opportunities present risks to
student success
☐
6
(0)
There is evidence of inaccuracy in the
course content.
☐
6
(0)
Some clarity or consistency issues with Course directions present significant
the course directions are present
clarity or consistency issues that
would impede student success
(4.5)
Content underwent review to ensure
accuracy and contributors were
appropriately qualified and
credentialed
☐
☐
(4.5)
Directions provided to students and
instructors are clear and consistent
☐
Some course activities, materials, or
workload do not reflect appropriate
difficulty and complexity for the level
of the course
3
(0)
Page 4 of 8
Polish
QL
Accessibility
Course embodies the professionalism
and polish of the institution and
effectively promotes the brand
There are no noticeable errors or
unprofessional elements
☐
☐
(3)
(2)
There are minor but noticeable errors There are significant errors or
or unprofessional elements
instances of unprofessionalism
☐
(1.5)
☐
3
(0)
Course materials are exceptionally
Course materials cost less than $100 Course materials cost more than $100 Course materials are unreasonably
affordable and fully accessible to almost and are fully accessible to a
or present minor accessibility issues
expensive given available options or
any type of student
reasonable majority of students
present major accessibility issues
QL
Tools and
Technology
QL
☐
(3)
☐
(2)
Course fully integrates emerging tools
and technologies in a way that
significantly improves engagement
Course employs tools and
technologies to support meaningful
student learning
☐
☐
(3)
(2)
☐
(1.5)
Course does not employ appropriate
tools or technology
☐
(1.5)
☐
(0)
Course employs tools or technology
that present significant risks to
student success
☐
3
3
(0)
Page 5 of 8
Rubric Legend
Critical
Definition
Element
Course
These are the necessary terminal knowledge, skills,
Outcomes dispositions and abilities a student must
demonstrate in order to successfully pass the course.
Course outcomes should define and contextualize
the enduring student cognitive performance(s) and
qualify the essential criteria for successfully meeting
the outcome. Ideally, course outcomes would be
derived from the larger program outcomes and
vision.
Learning These are the specific, itemized goals describing the
Objectives various student performances and expectations of
the module. These should be more concrete,
measurable, and have fewer dimensions than the
course outcomes for purposes of accurate
measurements. Ideally, learning objectives are
derived from the course outcomes, and, as a whole,
represent a logical path to those terminal knowledge,
skills, and abilities that illustrates a trajectory in
student cognitive performance from the beginning of
the course to its conclusion.
Vehicle Vehicles are critical components of the pedagogical
approach. These are the activity types employed in
the course (ex. short papers, blogs, journals, wikis,
discussions, problem sets, etc.).
Relationship to Student Success
Because the course outcomes are the immutable parameters and requirements for the assessments, they are
considered foundational to the course’s essential nature and vision. Issues with the outcomes will either corrupt
or obfuscate that vision. This is highly problematic for course design, which requires making pedagogical
decisions based on that vision. Courses built on foundations that are not sound are far more likely to suffer
from pedagogical issues, such as being inappropriate in scope or rigor, containing “fluff” or busy work, not
properly preparing students for subsequent courses in the program, assessing the wrong constructs and
drawing the wrong inferences about student competency, or any number of other serious issues that threaten
student success. Additionally, proper course outcomes and alignment to those course outcomes are necessary
for transparently communicating expectations to students.
Learning objectives serve many purposes that are essential to student success. First, they are necessary for
transparent communication of expectations to students. Secondly, they allow for more accurate measurements
of student competency that, as a whole, can be used to substantiate having met the multi-dimensional,
enduring course outcomes. Finally, sound, aligned, and pedagogically appropriate outcomes are necessary for
scaffolding and supporting student mastery. Ideally, learning objectives allow for a concrete, but customizable
approach that provides students with the most logical path to successfully meeting the course outcomes. The
design of learning objectives can also be a technique to manage the conversation with the SME. In the process
of unpacking the learning objectives from the course outcomes, sequencing them into modules, and then
redrafting them to add learning context specific to a content domain or learning topic, the design is informed by
the desired student cognitive performances as opposed to solely by content consumption.
The deliberate selection of an activity vehicle based on the intended student experience or performance is
necessary for sound, coherent course design. When inappropriate vehicles are used, students are made to
engage in inappropriate or confusing experiences. Vehicles should be selectively chosen to serve the overall
pedagogical approach and not simply as a means for adding additional formative assessments in a module
which lend to an impression of “busywork.” For example, the use of short response prompts in a discussion
forum without language that directs students towards one another to actively engage in a dialogue is not
appropriate for the intent of the vehicle.
Cohesive The course contains the big picture, enduring topics, When the central themes of the course are appropriate, authentic, and engaging, they inform the design in such
Design
principles or questions that are threaded throughout a way that creates cohesion. Instead of siloing topics in disparate modules, students can find connections to
and bring a sense of cohesion to the course.
previous learning and build upon those foundations to develop mastery of the outcomes. For example, if the
big question of the course is “what is post/colonial?” then the course would contain an introduction that
introduces the question, the modules might revisit these question through various learning topics, and the final
module would close the loop by bringing students together in a “jigsaw strategy” collectively respond to the
question instead of introducing new content and seeking to formatively assess that content.
Introduction This is the “home page” of the course, in which the If students understand where a course fits within the larger scheme of their personal and professional goals,
salient course information (ex. course description,
they are more likely to believe in the relevance of the course. Additionally, a fully-fleshed out context is
course purpose, outcomes, connection to personal necessary for transparent communication of purpose and expectations to the student. A course introduction
and professional contexts, etc. special navigation
sets the stage for the student experience and sets the stage for threading together a cohesive narrative from
Page 6 of 8
issues, etc.) and learning path are advertised to the
student. See the appendix of this document for a
sample conceptual map.
Alignment Alignment is the extent to which the summative
assessments align to the course outcomes.
Alignment to course outcomes is essential for
comprehensive measurement and data mapping of
the course outcomes.
Authenticity Authenticity of assessment concerns the emulation
of real-world performances.
Summative
Grading
Metrics
Formative
Assessments
Formative
Grading
Metrics
Weight
Scaffolding
module to module.
As the course outcomes represent the required knowledge, skills, abilities, and dispositions (KSADs) students
must demonstrate before the completion of a given course, alignment of the summative assessment
instruments to the outcomes is essential. Tangential or non-existent alignment could result in measurement of
the wrong constructs, ill-preparing students for future courses and professional work.
Utilizing authentic methods of assessing student attainment of the KSADs will further prepare students for the
real world. If we are not measuring the KSADs in the course outcomes in an authentic manner, we are missing
the opportunity to adequately prepare students for long-term personal and professional success.
Methods of calculating numerical grades on
Valid and reliable grading metrics are necessary for ensuring proper quantification of student success.
summative assessments.
Summative assessments are critical aspects of a student’s grade, and the rubrics and scoring methods utilized
must be applicable to the task and adhere to psychometric best practices. Invalid and unreliable grading metrics
could misrepresent or impede student success.
These are low-stakes opportunities that scaffold to It is essential that student’s be afforded the opportunity to practice, learn from mistakes in a low-stakes
the summative assessments.
environment, and thereby build mastery toward the course outcomes before being summatively assessed in the
critical tasks. Otherwise, our course design is not providing students with the necessary scaffolding for success.
Methods of calculating numerical grades on
The grading metrics for formative assessments are necessary in measuring student success and competency
formative assessments.
throughout the duration of a course. Invalid and unreliable metrics can hinder successful scaffolding of KSADs
and ill-prepare students for success in summative assessments and future course work.
Weight is the distribution of graded components that The final course grade is the indication of student success within a course while the summative assessment is
make up the final course grade.
the indication of student competency in terms of the course outcomes; therefore, the summative assessments
should be critical in passing a course. The general rule for summative weight is about 30%-40% of the final
course grade. Beyond this, weight should be distributed appropriately depending on the complexity and level
of performance required in an assignment.
This is the extent to which the instructional materials Without providing the necessary instructional materials, we can hardly expect students to perform well on the
have direct relevancy to the assessments of the
assessments. Additionally, instructional materials that are not directly relevant to the assessments can serve as
course.
distractions that take away student focus from the salient KSAD.
An example of poor scaffolding of instructional materials includes the existence of an article or video that might
be relevant to the discipline, but is not a topic being covered in the course and thus does not have direct
relevancy to the assessments of the course. This could be confusing to the student, potentially impeding
student success.
An example of effective scaffolding of instructional materials includes the existence of a video on “Autism,”
when the Short Paper for that module prompts student to “describe some of the common characteristics and
needs of students with communication disorders and autism spectrum disorders. Outline some instructional
and environmental modifications, accommodations, and adjustments that could be implemented within a
school to meet the needs of these students.”
Workload This includes the appropriateness of the workload for It is important that student workload is appropriate and feasible for the level and complexity of the course. For
the level and complexity of the course.
instance, it would be inappropriate for a 200-level course, in a one week-long module, to ask students to read a
Page 7 of 8
Instructional This is the intelligibility and ease of consumption of
Clarity
all directions and instructional communications in
the course.
Content
Accuracy
Modalities
Organization
Automation
Polish
Accessibility
Tools and
Technology
300-page book, submit a short paper, submit a response to the discussion forum, and complete a milestone for
the final course project. Similarly, it is inappropriate for a 600-level course to simply have students read one
chapter of the course text and submit a response to the discussion forum.
If students do not understand the directions for the materials and activities of a course, they can hardly be
expected to accomplish those tasks successfully. This includes all directions and instructional communications
relative to all formative and summative assessments in the course, including the rubrics of each grading activity.
Some examples of problematic instructional clarity potentially impeding student success include: (1) When the
“Main Elements” section of a Final Paper Guidelines document asks students to “Choose one or more of the
following issues...” while the paragraph above it states “Choose an issues from the list below.” OR (2) if the link
to the homework assignment on the module page reads “Homework Rubric,” yet the title of the actual
document is “Problem Set.” OR (3) Basic instruction of an assessment can only be found on a module page, with
missed opportunities to provide further clarification on other documents, such as the assessment rubric, for
example.
This is the extent to which the content of the course In order for our courses to be defensible in terms of academic quality, it is necessary that their content reflect
reflects the most current and accepted body of
the most current and accepted body of knowledge in the field. The academic quality reputations of our courses
knowledge in the field.
are connected to student’s future successes in their personal or professional endeavors, in the form of the value
of their degree and their preparedness for real-world contexts.
This is the various formats in which instruction is
In order to present students with the maximum possible opportunities for demonstrating their mastery, the use
provided (ex. resource types, platforms, deliverable of multiple modalities is required. In this way, non-essential variables can be manipulated in order to isolate and
types, etc.) that allow for multiple learning styles to best measure relevant constructs.
be addressed and actively engaged.
When using multiple modalities, we are more likely to reach students and “hone-in” on their abilities to learn.
This includes incorporating various assessments and modalities with the use of discussion forums, quiz/test
taking as formative assessments, using visual imagery, such as relevant video clips, reading scholarly articles to
complement the text-based reading, short papers, journals, blogs, etc.
This is the schema used to assign relationships and
This critical element speaks to the extent to which the course content is organized in a logical, intuitive way and
build navigability between the different components lets students focus on the course content instead of the locations of that content.
of the course.
The use of technology for grading and providing
When courses employ appropriate, automated assessments, instructor time is freed up to provide more
immediate feedback to students.
meaningful instruction and guidance to students in other assessments. Additionally, students are able to utilize
timely feedback that would otherwise not be available to them until after instructor processing.
The overall “look” of the course and technical
This area is a place to document any spacing, font, formatting, or grammatical errors. Though seemingly trivial,
accuracy of the communications with respect to
these errors could produce a negative perception of course quality that draws in to question the overall
standard conventions
professionalism of the university.
This is the ability for all students to reasonably
If there are unreasonable barriers to accessing the course materials, students can hardly be expected to
acquire and use the course materials (ex. cost of
successfully complete the tasks which make up their course grades.
materials, broken links)
SNHU bookstore - http://bookstore.mbsdirect.net/snhu.htm
These are any special applications, widgets, tools, or It is important that we strive to employ emerging tools and technology in our courses without giving in to the
other technology employed in the course.
temptation to use these for their own sake, which could detract from the course vision.
Page 8 of 8