Teacher & Principal Evaluation Choosing an Instructional Framework Danielson vs. UW Cel 5D’s + Scott Poirier spoirier@washingtonea.org K-12 Education and Policy Issues Coordinator Three Instructional Frameworks Marzano Research based on meta-analysis, control studies, correlation studies, focus group feedback, observations UW 5 Ds + Research based on UW work, Classroom practice and principal observation Danielson Marzano Danielson Research based on classroom-based observations, theorizing teaching practice, MET project study, focus group feedback, cross walk with student assessment U.W CEL 5D+ Objective and Logical Analysis of the Two Remaining Frameworks Architecture of the two Research-Based Instructional Frameworks Domains Dimensions Scored Components Elements Danielson 5 D+ 4 Domains 7 Dimensions 22 + 5 37 + 5 76 0 Questions to ask Are there any major differences in the architecture of the two frameworks? If so, what? Will the differences in the architecture impact how we will operationalize the evaluation system? Will there be a difference in the amount of time it would take to use one framework over the other? Review the Framework Components Conduct an initial “circle” process where participants review the frameworks through the lens of their district culture. Use the crosswalk to go down each criterion and circle the concepts that most align with the district’s philosophy. Underline interesting observations. Questions to ask • Do the frameworks have a vocabulary aligned to past district practice and support? • Do the frameworks cover the “really big deals” that the district finds important in a teacher/principal evaluation? Frameworks missing “really big deals?” • Does one framework language lend itself better to the type of evidence the district will collect in the process? • Specificity? Student or Teacher centric? Jargon? Clear language? What are Your Observations - Component Level Danielson 5 D’s + Architecture of the Two Research-Based Instructional Frameworks Domains Dimensions Scored Components Elements Floor Gradations Ceiling Cut Line Time Support Available? History? Inter-rater reliability? Danielson 5 Ds + 4 Domains 7 Dimensions 22 +5 37 + 5 76 0 The Floor What is a “1” You may want to dissect the actual rubric language for each of the instructional frameworks. One of the biggest differences may be what constitutes a “1” (the floor). You usually see four different types of “floors.” 1. 2. 3. 4. Language that uses detrimental terms, practice is detrimental to students No impact on students – absence of content/process/learning/ or using strategy wrong Minimal impact on students (words like rarely, minimal, few, etc) Progressions (North Carolina, McCREL) Rubric 1 Rubric 2 When the strategy is called for the teacher does not use it or the teacher uses the strategy incorrectly or with parts missing. Teacher Interaction with at least some students is negative, demeaning, sarcastic or inappropriate to the age or culture of the students. Students exhibit disrespect for the teacher Rubric 3 Students are rarely given an opportunity to assess their own learning in relation to the success criteria. Can you identify the floor that represents Danielson and 5D’s +? The Floor What is a “1” 1. 2. 3. 4. Language that uses detrimental terms, practice is detrimental to students No impact on students – absence of content/process/learning/ or doing strategy wrong Minimal impact on students (words like rarely, minimal, few, etc) Progressions (Central Valley in Spokane, North Carolina, McCREL) Questions 1. What are the implications/issues related to the different types of floors? 2. Will the different “floors” impact the types of measures and evidence collected? 3. Would there be any difference how a district approaches professional development based on the different floors? Gradations Differentiated language - What kind of language differentiates the levels. Basically there are three kinds of gradations. Quantitative, qualitative, and progressions. You may want to have the discussion regarding the differences and what your team thinks will work the best in your district 1. Quantitatively (never, few, sometimes, consistently, frequently, always, etc) 2. Qualitatively (describes differences in detail and quality) 3. Progressions (do one thing, next level do two things, next level do three things, etc) 4. Combination of the three above Example 1 below Student Engagement: Work of High Cognitive Demand Unsatisfactory Teacher expectations and strategies engage few students in work of high cognitive demand. Basic Teacher expectations and strategies engage some students in work of high cognitive demand. Proficient Distinguished Teacher expectations and strategies engage most students in work of high cognitive demand. Teacher expectations and strategies engage all students in work of high cognitive demand. Gradations 1. Quantitatively (never, few, sometimes, consistently, frequently, always, etc) 2. Qualitatively (describes differences in detail) 3. Progressions (do one thing, next level do two things, next level do three things, etc) 4. Combination of the three above Example 2 below Creating an Environment of Respect and Rapport Unsatisfactory Basic Proficient Distinguished Classroom interactions, both between the teacher and students and among students, are negative, inappropriate, or insensitive to students’ cultural backgrounds, and characterized by sarcasm, put-downs, or conflict. Classroom interactions, both between the teacher and students and among students, are generally appropriate and free from conflict, but may be characterized by occasional displays of insensitivity or lack of responsiveness to cultural or developmental differences among students. Classroom interactions, both between teacher and students and among students, are polite and respectful, reflecting general warmth and caring, and are appropriate to the cultural and developmental differences among groups of students. Classroom interactions among the teacher and individual students are highly respectful, reflecting genuine warmth and caring and sensitivity to students’ cultures and levels of development. Students themselves ensure high levels of civility among members of the class. Gradations 1. Quantitatively (never, few, sometimes, consistently, frequently, always, etc) 2. Qualitatively (describes differences in detail) 3. Progressions (do one thing, next level do two things, next level do three things, etc) 4. Combination of the three above Review both Instructional frameworks and identify which framework uses which kind of gradations. Questions What are the implications/issues of using the different types of gradations? How will the different gradations impact the types of measures and evidence collected? Would there be any difference how a district approaches professional development based on the different types of gradations? How about with calibration and inter-rater reliability training? The Ceiling . . . Is it Achievable? The ceiling also has differences and districts should review them in detail to make sure the intent of their evaluation system is conveyed. Look for terms that are absolute (all, every, always, etc) and ask if the wording is “achievable” or is the wording nearly impossible for a principal and teacher to reach? What should a “4” be in your district? Rubric 1 Students always assess their own learning in relation to the success criteria and can always determine where they are in connection to the standard and proficiency. Rubric 2 The teacher adapts or creates new strategies to meet the specific needs of students for whom the typical application of strategies does not produce the desired effect. Rubric 3 High levels of student energy and teacher passion for the subject create a culture for learning in which everyone shares a belief in the importance of the subject and all students hold themselves to high standards of performance—for example, by initiating improvements to their work. Can you identify which is the Danielson framework and which is 5D’s+? Thoughts? Questions? Reflections? Levels of Performance - Danielson Unsatisfactory Lack of Unsafe Harmful Unclear Unaware Poor Unsuitable Basic Inconsistent Partial General Attempts Awareness Moderate Minimal Proficient Consistent Frequent Successful Appropriate Clear Positive Smooth Teacher Directed Success Distinguished Solid Seamless Subtle Skillful Preventative Leadership Students Student Directed Success Levels of Performance – 5D’s + Unsatisfactory Basic Rarely Limited Does not Occasionally Few Some Non-existent Does not effectively Absent Poorly executed Proficient Frequently Most Teacher Directed Success Distinguished Always All Student Directed Success The Cut Line: Basic “Deemed Satisfactory” Review each rubric with the cut line in mind: Basic for the first 5 years Proficient for more than 5 years Rubric 1 The lesson is based on grade level standards and the learning target(s) align to the standard. The lesson is occasionally linked to broader purpose or a transferable skill. Rubric 2 The classroom culture is characterized by little commitment to learning by teacher or students. The teacher appears to be only going through the motions, and students indicate that they are interested in completion of a task, rather than quality. The teacher conveys that student success is the result of natural ability rather than hard work; high expectations for learning are reserved for those students thought to have a natural aptitude for the subject. The Cut Line: Proficient Review each rubric with the cut line in mind: Basic for the first 5 years Proficient for more than 5 years Rubric 1 The lesson is based on grade level standards and the learning target(s) align to the standard. The lesson is frequently linked to broader purpose or a transferable skill. Rubric 2 The classroom culture is a cognitively busy place where learning is valued by all, with high expectations for learning being the norm for most students. The teacher conveys that with hard work students can be successful. Students understand their role as learners and consistently expend effort to learn. Classroom interactions support learning and hard work. Mount Baker TPEP Evaluation Tool for Each of the 8 Criteria Part Two: Framework Language is … Marzano 5D Danielson 4 4 3 3 3 3 3 2 3 2 3 3 Marzano 5D Danielson 1. leads to instructional improvement rather than the creation of time consuming products 3 2 3 2. leads to instructional improvement rather than compliance activities 2 2 2 3. balances sophistication in understanding instruction with practical understandings of workload and environment 4. promotes teacher feelings of safety 5. values reflection 6. promotes professional growth 7. can realistically be fully implemented in the district (scale-able) 8. Aligns with district values 2 2 2 2 3 3 1 2 3 3 1 2 3 3 1 30 26 27 1. easy to understand 2. objective 3. consistent 4. applicable to a variety of teaching assignments Rating scale is 1-4 following the U B P D model. Notes: Part three: The Framework process… Rating scale is 1-4 following the U B P D model. Notes: Total for Criteria 3 Student growth Rubric Language Criterion 3 SG 3.1 Establishes appropriate student growth goals for subgroups of students not reaching full learning potential. Goals identify multiple, high-quality sources of data to monitor, adjust, and evaluate achievement of goals. Criterion 6 SG 6.1 Establishes appropriate student growth goals for whole classroom. Goals identify multiple, high quality sources of data to monitor, adjust, and evaluate achievement of goals. SG 3.2 & SG 6.2 - Multiple sources of growth or achievement data from at least two points in time show clear evidence of growth for most students. Criterion 8 SG 8.1 Consistently and actively collaborates with other grade-level, subject matter or instructional team members to establish goals, to develop and implement common, highquality measures, and to monitor growth and achievement during the year. Time How much time will frameworks take to score? How many components will need to be scored? Danielson will score 22 components. 5D’s + will score 37 components. It’s a consideration regarding how much time the principal and teacher evaluation will logistically take when it’s operationalized. . . But are all components created equal? Policy decisions vs. what does it look like in the trenches. Kent School District History of the District Supporting Instructional Frameworks • What kind of resources have already been used for supporting an instructional framework in your district? • What kind of support has been provided? • How much professional development has been conducted? • If there is an instructional framework culture in the district, what is the saturation level among principals and teachers? (operationalized?) • Is there any negative or positive history already with one or more instructional framework? Which ones? • Any ramifications choosing one framework over the other? If so, what are they? • What are others considerations that need to be brought up connected to the history of the district and instructional frameworks? Support, PD, Manage the System • What kind of professional development support is available for each framework? Technical support? • What are the costs? • Is it supported by ESDs? • Can you join forces with another nearby district using the same framework to mitigate costs (bringing in an author/workshops, etc?) • Inter-rater reliability What have we looked at so far? What more do you need? • • • • • • • • • • • Research Architecture Scored components Floor Ceiling Gradations Cut line Time Support Inter-rater reliability & calibration History of the district • • • • • • • Language differences Teacher centric Student centric Specificity Professional development Measures and evidence And more Decision Time Question: As you think about the two instructional frameworks, where would you rate each one on a scale between 1 and 10 (with 10 being high) as it relates to being the best fit for your district? 1 2 3 4 5 6 All of us are smarter than any one of us 7 8 9 10