Aligning the Quality Matters Rubric to the
Technological, Pedagogical Content Knowledge
Conceptual Framework
Cheryl Ward
University of Akron
USA
cward@uakron.edu
Abstract: This paper is a report describing a process used to align the Quality Matters
Rubric to the TPACK (Technological, Pedagogical Content Knowledge) conceptual
framework and implications for its use. Preliminary data will be shared on a study that
indicates use of the QM rubric can increase technological, pedagogical content knowledge
in rubric users.
Introduction
The use of technology for learning has been a rocky road for most educators in K-12 and higher
education who may have limited experience with technology and how it can be used to support certain content
and pedagogical decisions. Decisions about technology use can be void of connections with content or
pedagogical decisions and based on availability or pressure to conform to use trends.
Many professional development efforts related to technology integration—including educational
technology courses in many teacher education programs—focus primarily on the affordances and
constraints of particular technological tools and resources. While these learning opportunities often
challenge teachers to consider how particular tools and resources may best be leveraged in particular
content areas and learning environments, the primary focus remains on the technology, rather than on
curriculum-based learning needs. (Hofer & Harris, 2010).
TPACK (Technological, Pedagogical, Content Knowledge) is a conceptual framework that explains the
complex, multifaceted and contextual nature of teacher knowledge of the relationship between technology,
content and pedagogy (Mishra & Koehler, 2006). Schulman (1986) wrote of specialize knowledge that teachers
develop at the intersection of content and pedagogy (PCK) and Mishra and Koehler expanded on this with the
introduction of technological pedagogical content knowledge (TPCK). The TPACK framework outlines overlap
areas that explain complementary knowledge that has to be developed in order to integrate technology in a way
that supports content and pedagogical decisions. In the figure below and in their seminal work Mishra and
Koehler illustrate the discrete domains of Technological Knowledge (TK), Pedagogical Knowledge (PK), and
Content Knowledge (CK) and the overlapping areas representing area of combined new knowledge:
Pedagogical/Content Knowledge (PCK), Technological/Content Knowledge (TCK), Technological/Pedagogical
Knowledge (TPK), and the middle overlap of all three circles called TPCK-Technological, Pedagogical, Content
knowledge.
TPACK has emerged as a useful conceptual framework for understanding the teachers’ knowledge base
needed for effectively teaching with technology (Voogt,et.al.,2011). In the development of online learning the
technology typically becomes the first topic of discussion, but TPACK discussions have clearly helped
instructors think about pedagogical and content organizational changes that are relevant in this work.
Figure 1. Technological, Pedagogical, Content Knowledge (Mishra & Koehler, 2011)
Archambalt and Oh-Young (2009) examined how teachers prepare to teach in online environments.
They found that the three major components (technology, content, and pedagogy) needed to ensure quality
instruction were well expressed in the TPACK conceptual framework. In their investigation of key issues
specific to online teaching, they found the TPACK framework was particularly useful.
The TPACK framework has allowed instructors of online learning to view each component in an
intentional way reflecting on their own knowledge of technology, content and pedagogy, but for some instructors
a new look at these areas in an overlapping context can be challenging. Mishra and Koehler (2006) write about
this new awareness for instructors working in online environments:
For instance, consider faculty members developing online courses for the first time. The relative
newness of the online technologies forces these faculty members to deal with all three factors, and the
relationships between them, often leading them to ask questions of their pedagogy, something that they
may not have done in a long time” (p. 1030).
For this study gaining an understanding of how technological, pedagogical content knowledge
development could be impacted by a concurrent process instructors use in the development of online learning
was a goal. A process of peer review of online coursework called Quality Matters (QM) is used to certify the
quality of the design of online and blended courses. QM process consists of a rubric and professional
development to help instructors develop high quality online courses using technology to support learning. The
professional development training focuses on the participants’ ability to describe the Quality Matters process,
describe the QM rubric, apply the eight general QM review standards and 41 specific standards to course
development, and to develop a quality online course that meets all QM required standards. The QM rubric has a
scoring system and set of online tools that help facilitate the course evaluation by the review team (Quality
Matters, 2011). Quality matter emphasizes that the rubric specifically focuses on course design, rather than on
course delivery or course academic content. The rubric includes 7 critical online course components-1) course
overview and introduction; 2) learning objectives; 3) assessment and measurement; 4) resources and materials;
5) learner engagement; 6) course technology; and 7) accessibility.
The Study
This paper reports on one piece of a larger study looking at if the use of the 2008-2011 Higher Education
QM Rubric would impact the development of technological, pedagogical, content knowledge in instructors. The
first step of this study and the focus of this paper is on the alignment work done with the QM Rubric and the
TPACK conceptual framework. One of the initial premises for this study was that although QM states that the
rubric specifically focuses only on course design not delivery or content, it has become clear in understanding
TPACK that it is hard to talk about one without the others due to the dynamic nature of these knowledges. These
are also not separate topics when instructors discuss this process. They are overlapping dynamic pieces of a
puzzle that fit together through an understanding of how each piece operates separately and more importantly
together to help instructors develop this new type of knowledge. Understanding the dynamic relationship
between the QM Rubric and TPACK was a foundational piece of the main study.
Aligning the QM rubric with the TPACK conceptual framework began with a process of analyzing each
QM rubric item and considering what area of TPC knowledge might be either developed or enhanced based on
the rubric area. For example, the QM rubric item 6.2 The tools and media support student engagement and
guide the student to become an active learner was aligned to TPK which is the knowledge of the technologypedagogy interaction --independent of a topic-specific representation or content-specific instructional strategy.
So theoretically as instructors are involved in learning and applying the QM rubric they are also growing in
different areas of TPACK. Growth is facilitated through their own reflection as well as peer discussion and
collaborations around the application of the rubric.
Each of the 41 rubric items were aligned with one specific TPACK area including the 3 single areas of
CK (content knowledge), PK (pedagogical knowledge), and TK (technology knowledge), the overlapping areas
of PCK (pedagogical content knowledge), TCK (technological content knowledge), and TPK (technological
pedagogical knowledge) and the main “sweet spot” TPCK (technological, pedagogical content knowledge.
Three professors well-versed in TPACK and in the QM rubric engaged 25 graduating master’s students
and graduate assistants in the process to align each individual rubric item with the areas of TPACK. Each of the
TPACK areas had a common description that was provided to the reviewers and is included in the Rubric
Alignment Document. Each participate coded the rubrics by highlighting the sections that referred to technology,
pedagogy and content by color. These results were summarized and discrepancies were discussed by the
professors and resolved. The Rubric Alignment Document is shared in the Appendix.
Findings
The goal of this part of the research was to create a process for the alignment work and to discover how
the QM rubric looked when matched with the sections of TPACK. Participants in this process were asked to
comment on the process and most described it as a “natural, easy process”. The following are a few select
comments from alignment participants:
“It was easy to align these rubric items to the TPACK areas because our training has shown us how to think
about the 3 areas of technology, pedagogy and content. These are easy to code because we have a clear
understanding of these areas.”
“Everything did not always fit neatly into the TPACK categories, but then I would go back and read the section
description and think about what that would look like in a course or look like as it was developed and then I had
a greater sense of what to choose.”
“QM and TPACK fit together to create a richer picture of the processes and knowledge involved in using
technology to develop online learning. It seemed fairly obvious which rubric items fit into the TPACK areas,
especially since TPACK provides the overlap areas which most of the rubric items are.”
Figure 2 shows that the QM Rubric is well aligned with the TPACK conceptual framework. Note that
the most highly aligned area was PK or Pedagogical Knowledge showing that pedagogical knowledge is highly
aligned with the design and development of quality online learning
Figure 2. Number of QM Rubric items aligned with each area of TPACK
The second highest area was PCK or Pedagogical Content Knowledge—content is important of course.
The third area TPK or Technological Pedagogical Knowledge indicates that in the design of online learning
understanding how technology supports certain pedagogical decisions is paramount! It was interesting that a
rubric that purports to only address the design elements of an online class align so highly with the pedagogical
elements in the TPACK conceptual framework. This alignment supports initial predictions that the elements of
the QM Rubric foster discussion and knowledge development in more areas than just design of these
environments. Technological, pedagogical and content discussions are overlapping and connected in a dynamic
way that result in inability to discuss or work on them in isolation for online course development.
Conclusions
The alignment of the 2008-2011 QM Rubric with the TPACK areas was instrumental in establishing that
use of the rubric contributed to gains in technological pedagogical content knowledge in instructors who
developed learning using the rubric. The process for aligning the rubric items with the TPACK areas proved to
be reliable and useful in understanding how the use of the rubric might impact TPACK growth. The full study
will be reported in a future paper, but preliminary data showed that development of online learning as measured
by the QM rubric was significantly improved post rubric implementation and instruction.
References
Archambault, L. & Oh-Young, C. (2009). Putting the T in PCK: Exploring the Nature of the TPACK Framework
Among K-12 Online Educators Using a Web-Based Survey. In I. Gibson et al. (Eds.), Proceedings of Society for
Information Technology & Teacher Education International Conference 2009 (pp. 4008-4014). Chesapeake,
VA: AACE.
Bauer, J. & Kenton, J. (2005). Toward Technology Integration in the Schools: Why It Isn’t Happening. Journal of
Technology and Teacher Education, 13(4), 519-546. Norfolk, VA: AACE.
Hofer, M. & Harris, J. (2010). Differentiating TPACK Development: Using Learning Activity Types with Inservice and
Preservice Teachers. In D. Gibson & B. Dodge (Eds.), Proceedings of Society for Information Technology &
Teacher Education International Conference 2010 (pp. 3857-3864). Chesapeake, VA: AACE.
Koehler, M., & Mishra, P. (2005). What happens when teachers design educational technology? The development of
technological pedagogical content knowledge. Journal of Educational Computing Research, 32(2), 131-152
Mishra, P., & Koehler, M. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge.
Teachers College Record, 108(6), 1017-1054.
Mishra, P., & Koehler, J. (2011). TPACK Image, www.tpack.org.
Quality Matters (2011). What is the QM Program? Retrieved March 30, 2011, from
http://www.qmprogram.org/welcome
Shulman, L. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15 (2), 4-14.
Voogt, J., Shin, T., Mishra, P., Koehler, M., Schmidt, D., Baran, E., Thompson, A., Wang, W., Alayyar, G., Fisser, P.,
Agyei, D., Ormel, B., Velthuis, C., Tondeur, J. & Gibson, D. (2011). Teachers’ assessment of TPACK: Where
are we and what is needed?. In M. Koehler & P. Mishra (Eds.), Proceedings of Society for Information
Technology & Teacher Education International Conference 2011 (pp. 4422-4426). Chesapeake, VA: AACE
Appendix
Rubric Alignment Document
Alignment of the QM Rubric Standards within the TPACK conceptual framework
This document is initial work in the alignment of each rubric standard within the conceptual framework.
This alignment shows how each of the rubric items contributes to an instructor/developers growing knowledge in the
areas of Technology, Pedagogy, and Content knowledge as well as the dynamic overlap areas indicated in the diagram.
TPACK research posits that the use of technology in teaching and learning is a complex and wicked problem that required
new sets of knowledge to be developed by instructors especially in the areas of TPK, TCK and TPCK. The use of technology
for learning and teaching has typically been viewed as a simple task of integration, but we find that it is a much more
complex set of knowledge and skills.
Professional development using the concepts of Quality Matters intends to focus only on course design. Our research
shows that rich discussions in the areas of content, pedagogy, and technology result from the application of the QM rubric
and professional development. Discussion in these areas is a core part of creating a quality online course. Although
course design is important, discussion around TPCK becomes core to the establishment of a quality online course.
TPCK
a knowledge of the
technology-pedagogy-content
interaction in the context of
content-specific instructional
strategies
1.2 A statement introduces the students to the purpose of the course and to its components; in the case of hybrid course, the statement clarifies the
relationship between the face-to-face and online components. TPCK
6.6 Instructions on how to access resources at a distance are sufficient and easy to understand. TPCK
6.7 The course design takes full advantage of available tools and media. TPCK
7.4 Course instructions answer basic questions related to research, writing, technology, etc., or link to tutorials or other resources that provide the
information. TPCK
8.3 Course pages have links that are self-describing and meaningful. TPCK
TPK : a knowledge of the
1.3 Etiquette expectations (sometimes called “netiquette”) for online discussions, email, and other forms of communication are stated clearly. TPK
1.4 The self-introduction by the instructor is appropriate and available online. TPK
6.2 The tools and media support student engagement and guide the student to become an active learner.TPK
6.5 The course components are compatible with current standards for deliver modes.TPK
8.1 The course incorporates ADA standards and reflect conformance with institutional policy regarding accessibility in online and hybrid courses.TPK
8.2 Course pages and course materials provide equivalent alternatives to auditory and visual content. TPK
8.4 The course ensures screen readability. TPK
6.1 The tools and media support the learning objectives, and are appropriately chosen to deliver the content of the course. TCK
6.3 Navigation throughout the online components of the course is logical, consistent, and efficient.TCK
6.4 Students have ready access to the technologies required in the course.TCK
6.6 Instructions on how to access resources at a distance are sufficient and easy to understand. TCK
technology-pedagogy interaction
independent of topic-specific
representations or content-specific
instructional strategies.
TCK a knowledge of the
technology-content interaction
independent of pedagogy.
PCK combines knowledge of
activities and knowledge of
representations in order to
facilitate student learning. The
knowledge of pedagogical
activities here is content-specific
rather than general because PCK
is situated in a particular subject
area
CK a knowledge of the possible
topic- specific representations in a
given subject area.
PK to focus on a teacher’s
knowledge of the general
pedagogical activities that might
utilized
TK knowledge of how to use
emerging technologies.
Course Management
1.2 A statement introduces the students to the purpose of the course and to its components; in the case of hybrid course, the statement clarifies the
relationship between the face-to-face and online components.
2.5 The learning objectives are appropriately designed for the level of the course.PCK
3.1 The types of assessments selected measure the stated learning objectives and are consistent with course activities and resources. PCK
3.4 The assessment instrument selected are sequenced, varied, and appropriate to the content being assessed. PCK
4.1 The instructional materials contribute to the achievement of the stated course and module/unit learning activities.PCK
4.4 All resources and materials used in the course are appropriately cited. PCK
5.1 The learning activities promote the achievement of the stated course and module/unit learning objectives. PCK
7.2 Course instruction articulate or link to an explanation of how the instructor’s academic support system can assist the student in effectively using
the resources provided. PCK
7.3 Course instructions articulate or link to an explanation of how the institution’s student support services can help students reach their educational
goals, PCK
1.6 Minimum student preparation, and, if applicable, prerequisite knowledge in the discipline are clearly stated. CK
4.3 The instructional materials have sufficient breadth, depth, and currency for the student to learn the subject. CK
1.1 Instructions make clear how to get started and where to find various course components. PK
1.5 Students are asked to introduce themselves to the class. PK
2.1 The course learning objectives describe outcomes that are measurable. PK
2.2 the module/unit learning objectives describe outcomes that are measurable and consistent with the course-level objectives. PK
2.3 All learning objectives are stated clearly and written from student’s perspectives. PK
2.4 Instructions to students on how to meet the learning objectives are adequate and stated clearly. PK
3.2 The course grading policy is stated clearly.PK
3.5 “Self-check” or practice assignments are provided, with timely feedback to students. PK
4.2 The relationship between the instructional materials and learning activities is clearly explained to the student.PK
5.2 Learning activities foster instructor-student, content-student, and if appropriate the course, student-student interaction. PK
5.3 Clear standards are set for instructor responsiveness and availability (turn-around time for email, grade posting, etc.) PK
5.4 The requirement for student interaction are clearly articulated. PK
7.1 The course instructions articulate or link to clear description of the technical support offered. PK
1.7 Minimum technical skills expected of the student are clearly stated. TK
6.7 The course design takes full advantage of available tools and media. TK
8.2 Course pages and course materials provide equivalent alternatives to auditory and visual content. TK
8.4 The course ensures screen readability. TK
3.3 Specific and descriptive criteria are provided for the evaluation of students’ work and participation.
7.2 Course instructions articulate or link to an explanation of how the institution’s student support services can help students reach their educational goals.
7.3 Course instructions articulate or link to an explanation of how the institution’s academic support system can assist the student in effectively using the
resources provided.
8.1 The course incorporates ADA standards and reflect conformance with institutional policy regarding accessibility in online and hybrid courses.