Aligning the Quality Matters Rubric to the Technological, Pedagogical Content Knowledge Conceptual Framework Cheryl Ward University of Akron USA cward@uakron.edu Abstract: This paper is a report describing a process used to align the Quality Matters Rubric to the TPACK (Technological, Pedagogical Content Knowledge) conceptual framework and implications for its use. Preliminary data will be shared on a study that indicates use of the QM rubric can increase technological, pedagogical content knowledge in rubric users. Introduction The use of technology for learning has been a rocky road for most educators in K-12 and higher education who may have limited experience with technology and how it can be used to support certain content and pedagogical decisions. Decisions about technology use can be void of connections with content or pedagogical decisions and based on availability or pressure to conform to use trends. Many professional development efforts related to technology integration—including educational technology courses in many teacher education programs—focus primarily on the affordances and constraints of particular technological tools and resources. While these learning opportunities often challenge teachers to consider how particular tools and resources may best be leveraged in particular content areas and learning environments, the primary focus remains on the technology, rather than on curriculum-based learning needs. (Hofer & Harris, 2010). TPACK (Technological, Pedagogical, Content Knowledge) is a conceptual framework that explains the complex, multifaceted and contextual nature of teacher knowledge of the relationship between technology, content and pedagogy (Mishra & Koehler, 2006). Schulman (1986) wrote of specialize knowledge that teachers develop at the intersection of content and pedagogy (PCK) and Mishra and Koehler expanded on this with the introduction of technological pedagogical content knowledge (TPCK). The TPACK framework outlines overlap areas that explain complementary knowledge that has to be developed in order to integrate technology in a way that supports content and pedagogical decisions. In the figure below and in their seminal work Mishra and Koehler illustrate the discrete domains of Technological Knowledge (TK), Pedagogical Knowledge (PK), and Content Knowledge (CK) and the overlapping areas representing area of combined new knowledge: Pedagogical/Content Knowledge (PCK), Technological/Content Knowledge (TCK), Technological/Pedagogical Knowledge (TPK), and the middle overlap of all three circles called TPCK-Technological, Pedagogical, Content knowledge. TPACK has emerged as a useful conceptual framework for understanding the teachers’ knowledge base needed for effectively teaching with technology (Voogt,et.al.,2011). In the development of online learning the technology typically becomes the first topic of discussion, but TPACK discussions have clearly helped instructors think about pedagogical and content organizational changes that are relevant in this work. Figure 1. Technological, Pedagogical, Content Knowledge (Mishra & Koehler, 2011) Archambalt and Oh-Young (2009) examined how teachers prepare to teach in online environments. They found that the three major components (technology, content, and pedagogy) needed to ensure quality instruction were well expressed in the TPACK conceptual framework. In their investigation of key issues specific to online teaching, they found the TPACK framework was particularly useful. The TPACK framework has allowed instructors of online learning to view each component in an intentional way reflecting on their own knowledge of technology, content and pedagogy, but for some instructors a new look at these areas in an overlapping context can be challenging. Mishra and Koehler (2006) write about this new awareness for instructors working in online environments: For instance, consider faculty members developing online courses for the first time. The relative newness of the online technologies forces these faculty members to deal with all three factors, and the relationships between them, often leading them to ask questions of their pedagogy, something that they may not have done in a long time” (p. 1030). For this study gaining an understanding of how technological, pedagogical content knowledge development could be impacted by a concurrent process instructors use in the development of online learning was a goal. A process of peer review of online coursework called Quality Matters (QM) is used to certify the quality of the design of online and blended courses. QM process consists of a rubric and professional development to help instructors develop high quality online courses using technology to support learning. The professional development training focuses on the participants’ ability to describe the Quality Matters process, describe the QM rubric, apply the eight general QM review standards and 41 specific standards to course development, and to develop a quality online course that meets all QM required standards. The QM rubric has a scoring system and set of online tools that help facilitate the course evaluation by the review team (Quality Matters, 2011). Quality matter emphasizes that the rubric specifically focuses on course design, rather than on course delivery or course academic content. The rubric includes 7 critical online course components-1) course overview and introduction; 2) learning objectives; 3) assessment and measurement; 4) resources and materials; 5) learner engagement; 6) course technology; and 7) accessibility. The Study This paper reports on one piece of a larger study looking at if the use of the 2008-2011 Higher Education QM Rubric would impact the development of technological, pedagogical, content knowledge in instructors. The first step of this study and the focus of this paper is on the alignment work done with the QM Rubric and the TPACK conceptual framework. One of the initial premises for this study was that although QM states that the rubric specifically focuses only on course design not delivery or content, it has become clear in understanding TPACK that it is hard to talk about one without the others due to the dynamic nature of these knowledges. These are also not separate topics when instructors discuss this process. They are overlapping dynamic pieces of a puzzle that fit together through an understanding of how each piece operates separately and more importantly together to help instructors develop this new type of knowledge. Understanding the dynamic relationship between the QM Rubric and TPACK was a foundational piece of the main study. Aligning the QM rubric with the TPACK conceptual framework began with a process of analyzing each QM rubric item and considering what area of TPC knowledge might be either developed or enhanced based on the rubric area. For example, the QM rubric item 6.2 The tools and media support student engagement and guide the student to become an active learner was aligned to TPK which is the knowledge of the technologypedagogy interaction --independent of a topic-specific representation or content-specific instructional strategy. So theoretically as instructors are involved in learning and applying the QM rubric they are also growing in different areas of TPACK. Growth is facilitated through their own reflection as well as peer discussion and collaborations around the application of the rubric. Each of the 41 rubric items were aligned with one specific TPACK area including the 3 single areas of CK (content knowledge), PK (pedagogical knowledge), and TK (technology knowledge), the overlapping areas of PCK (pedagogical content knowledge), TCK (technological content knowledge), and TPK (technological pedagogical knowledge) and the main “sweet spot” TPCK (technological, pedagogical content knowledge. Three professors well-versed in TPACK and in the QM rubric engaged 25 graduating master’s students and graduate assistants in the process to align each individual rubric item with the areas of TPACK. Each of the TPACK areas had a common description that was provided to the reviewers and is included in the Rubric Alignment Document. Each participate coded the rubrics by highlighting the sections that referred to technology, pedagogy and content by color. These results were summarized and discrepancies were discussed by the professors and resolved. The Rubric Alignment Document is shared in the Appendix. Findings The goal of this part of the research was to create a process for the alignment work and to discover how the QM rubric looked when matched with the sections of TPACK. Participants in this process were asked to comment on the process and most described it as a “natural, easy process”. The following are a few select comments from alignment participants: “It was easy to align these rubric items to the TPACK areas because our training has shown us how to think about the 3 areas of technology, pedagogy and content. These are easy to code because we have a clear understanding of these areas.” “Everything did not always fit neatly into the TPACK categories, but then I would go back and read the section description and think about what that would look like in a course or look like as it was developed and then I had a greater sense of what to choose.” “QM and TPACK fit together to create a richer picture of the processes and knowledge involved in using technology to develop online learning. It seemed fairly obvious which rubric items fit into the TPACK areas, especially since TPACK provides the overlap areas which most of the rubric items are.” Figure 2 shows that the QM Rubric is well aligned with the TPACK conceptual framework. Note that the most highly aligned area was PK or Pedagogical Knowledge showing that pedagogical knowledge is highly aligned with the design and development of quality online learning Figure 2. Number of QM Rubric items aligned with each area of TPACK The second highest area was PCK or Pedagogical Content Knowledge—content is important of course. The third area TPK or Technological Pedagogical Knowledge indicates that in the design of online learning understanding how technology supports certain pedagogical decisions is paramount! It was interesting that a rubric that purports to only address the design elements of an online class align so highly with the pedagogical elements in the TPACK conceptual framework. This alignment supports initial predictions that the elements of the QM Rubric foster discussion and knowledge development in more areas than just design of these environments. Technological, pedagogical and content discussions are overlapping and connected in a dynamic way that result in inability to discuss or work on them in isolation for online course development. Conclusions The alignment of the 2008-2011 QM Rubric with the TPACK areas was instrumental in establishing that use of the rubric contributed to gains in technological pedagogical content knowledge in instructors who developed learning using the rubric. The process for aligning the rubric items with the TPACK areas proved to be reliable and useful in understanding how the use of the rubric might impact TPACK growth. The full study will be reported in a future paper, but preliminary data showed that development of online learning as measured by the QM rubric was significantly improved post rubric implementation and instruction. References Archambault, L. & Oh-Young, C. (2009). Putting the T in PCK: Exploring the Nature of the TPACK Framework Among K-12 Online Educators Using a Web-Based Survey. In I. Gibson et al. (Eds.), Proceedings of Society for Information Technology & Teacher Education International Conference 2009 (pp. 4008-4014). Chesapeake, VA: AACE. Bauer, J. & Kenton, J. (2005). Toward Technology Integration in the Schools: Why It Isn’t Happening. Journal of Technology and Teacher Education, 13(4), 519-546. Norfolk, VA: AACE. Hofer, M. & Harris, J. (2010). Differentiating TPACK Development: Using Learning Activity Types with Inservice and Preservice Teachers. In D. Gibson & B. Dodge (Eds.), Proceedings of Society for Information Technology & Teacher Education International Conference 2010 (pp. 3857-3864). Chesapeake, VA: AACE. Koehler, M., & Mishra, P. (2005). What happens when teachers design educational technology? The development of technological pedagogical content knowledge. Journal of Educational Computing Research, 32(2), 131-152 Mishra, P., & Koehler, M. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record, 108(6), 1017-1054. Mishra, P., & Koehler, J. (2011). TPACK Image, www.tpack.org. Quality Matters (2011). What is the QM Program? Retrieved March 30, 2011, from http://www.qmprogram.org/welcome Shulman, L. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15 (2), 4-14. Voogt, J., Shin, T., Mishra, P., Koehler, M., Schmidt, D., Baran, E., Thompson, A., Wang, W., Alayyar, G., Fisser, P., Agyei, D., Ormel, B., Velthuis, C., Tondeur, J. & Gibson, D. (2011). Teachers’ assessment of TPACK: Where are we and what is needed?. In M. Koehler & P. Mishra (Eds.), Proceedings of Society for Information Technology & Teacher Education International Conference 2011 (pp. 4422-4426). Chesapeake, VA: AACE Appendix Rubric Alignment Document Alignment of the QM Rubric Standards within the TPACK conceptual framework This document is initial work in the alignment of each rubric standard within the conceptual framework. This alignment shows how each of the rubric items contributes to an instructor/developers growing knowledge in the areas of Technology, Pedagogy, and Content knowledge as well as the dynamic overlap areas indicated in the diagram. TPACK research posits that the use of technology in teaching and learning is a complex and wicked problem that required new sets of knowledge to be developed by instructors especially in the areas of TPK, TCK and TPCK. The use of technology for learning and teaching has typically been viewed as a simple task of integration, but we find that it is a much more complex set of knowledge and skills. Professional development using the concepts of Quality Matters intends to focus only on course design. Our research shows that rich discussions in the areas of content, pedagogy, and technology result from the application of the QM rubric and professional development. Discussion in these areas is a core part of creating a quality online course. Although course design is important, discussion around TPCK becomes core to the establishment of a quality online course. TPCK a knowledge of the technology-pedagogy-content interaction in the context of content-specific instructional strategies 1.2 A statement introduces the students to the purpose of the course and to its components; in the case of hybrid course, the statement clarifies the relationship between the face-to-face and online components. TPCK 6.6 Instructions on how to access resources at a distance are sufficient and easy to understand. TPCK 6.7 The course design takes full advantage of available tools and media. TPCK 7.4 Course instructions answer basic questions related to research, writing, technology, etc., or link to tutorials or other resources that provide the information. TPCK 8.3 Course pages have links that are self-describing and meaningful. TPCK TPK : a knowledge of the 1.3 Etiquette expectations (sometimes called “netiquette”) for online discussions, email, and other forms of communication are stated clearly. TPK 1.4 The self-introduction by the instructor is appropriate and available online. TPK 6.2 The tools and media support student engagement and guide the student to become an active learner.TPK 6.5 The course components are compatible with current standards for deliver modes.TPK 8.1 The course incorporates ADA standards and reflect conformance with institutional policy regarding accessibility in online and hybrid courses.TPK 8.2 Course pages and course materials provide equivalent alternatives to auditory and visual content. TPK 8.4 The course ensures screen readability. TPK 6.1 The tools and media support the learning objectives, and are appropriately chosen to deliver the content of the course. TCK 6.3 Navigation throughout the online components of the course is logical, consistent, and efficient.TCK 6.4 Students have ready access to the technologies required in the course.TCK 6.6 Instructions on how to access resources at a distance are sufficient and easy to understand. TCK technology-pedagogy interaction independent of topic-specific representations or content-specific instructional strategies. TCK a knowledge of the technology-content interaction independent of pedagogy. PCK combines knowledge of activities and knowledge of representations in order to facilitate student learning. The knowledge of pedagogical activities here is content-specific rather than general because PCK is situated in a particular subject area CK a knowledge of the possible topic- specific representations in a given subject area. PK to focus on a teacher’s knowledge of the general pedagogical activities that might utilized TK knowledge of how to use emerging technologies. Course Management 1.2 A statement introduces the students to the purpose of the course and to its components; in the case of hybrid course, the statement clarifies the relationship between the face-to-face and online components. 2.5 The learning objectives are appropriately designed for the level of the course.PCK 3.1 The types of assessments selected measure the stated learning objectives and are consistent with course activities and resources. PCK 3.4 The assessment instrument selected are sequenced, varied, and appropriate to the content being assessed. PCK 4.1 The instructional materials contribute to the achievement of the stated course and module/unit learning activities.PCK 4.4 All resources and materials used in the course are appropriately cited. PCK 5.1 The learning activities promote the achievement of the stated course and module/unit learning objectives. PCK 7.2 Course instruction articulate or link to an explanation of how the instructor’s academic support system can assist the student in effectively using the resources provided. PCK 7.3 Course instructions articulate or link to an explanation of how the institution’s student support services can help students reach their educational goals, PCK 1.6 Minimum student preparation, and, if applicable, prerequisite knowledge in the discipline are clearly stated. CK 4.3 The instructional materials have sufficient breadth, depth, and currency for the student to learn the subject. CK 1.1 Instructions make clear how to get started and where to find various course components. PK 1.5 Students are asked to introduce themselves to the class. PK 2.1 The course learning objectives describe outcomes that are measurable. PK 2.2 the module/unit learning objectives describe outcomes that are measurable and consistent with the course-level objectives. PK 2.3 All learning objectives are stated clearly and written from student’s perspectives. PK 2.4 Instructions to students on how to meet the learning objectives are adequate and stated clearly. PK 3.2 The course grading policy is stated clearly.PK 3.5 “Self-check” or practice assignments are provided, with timely feedback to students. PK 4.2 The relationship between the instructional materials and learning activities is clearly explained to the student.PK 5.2 Learning activities foster instructor-student, content-student, and if appropriate the course, student-student interaction. PK 5.3 Clear standards are set for instructor responsiveness and availability (turn-around time for email, grade posting, etc.) PK 5.4 The requirement for student interaction are clearly articulated. PK 7.1 The course instructions articulate or link to clear description of the technical support offered. PK 1.7 Minimum technical skills expected of the student are clearly stated. TK 6.7 The course design takes full advantage of available tools and media. TK 8.2 Course pages and course materials provide equivalent alternatives to auditory and visual content. TK 8.4 The course ensures screen readability. TK 3.3 Specific and descriptive criteria are provided for the evaluation of students’ work and participation. 7.2 Course instructions articulate or link to an explanation of how the institution’s student support services can help students reach their educational goals. 7.3 Course instructions articulate or link to an explanation of how the institution’s academic support system can assist the student in effectively using the resources provided. 8.1 The course incorporates ADA standards and reflect conformance with institutional policy regarding accessibility in online and hybrid courses.