Mississippi Teacher Appraisal Tab 5 Principal Developers Judy Bassham Russell French George Malo June 2003 Revised June 2007 Revised 2009 ii TEACHER APPRAISAL Table of Contents MISSISSIPPI TEACHER APPRAISAL ...........................................................................................1 DATA SOURCES ...............................................................................................................................2 TEACHER PERFORMANCE STANDARDS.................................................................................3 STRUCTURED INTERVIEW .........................................................................................................8 Interview Scoring Form.............................................................................................11 Interview Scripting Form...........................................................................................17 Scoring Interview Responses.....................................................................................18 Key Concepts.............................................................................................................21 CLASSROOM OBSERVATION ...................................................................................................27 Pre-Observation Conference Record….…………….................................................32 Classroom Observation Record…..............................................................................33 Observation Analysis and Scoring Form….………...................................................34 Observation Scripting Form………………..………...................................................36 Definitions and Explanation of Teaching Practices....................................................37 DOCUMENT/ARTIFACT REVIEW ............................................................................................49 Document/Artifact Entry Form….……….................................................................51 Guidelines for Document/Artifact Entries….……….................................................52 Document/Artifact Scoring Rubrics….………..........................................................55 MISSISSIPPI TEACHER APPRAISAL REPORT ......................................................................59 Teacher Appraisal Report (cover sheet)….………...................................................65 Part A: Scores and Rationales.…………...................................................................66 Part B: Strengths, Weaknesses, Comments and Signatures.......................................70 Performance Level….…………………......................................................................71 Teacher Performance Standards Checklist.................................................................72 iii iv MISSISSIPPI TEACHER APPRAISAL Introduction The Mississippi Teacher Appraisal has been developed to evaluate teacher performance based on the Mississippi Teacher Performance Standards. These standards are defined by a series of indicators which, in turn, are defined by a series of definition items. Together, these three levels of appraisal criteria form the basis for measuring performance through the appraisal instruments. The structure is presented in Figure 1 below. Standard Indicator Indicator Definition Item Definition Item Definition Item Definition Item Figure 1: Structure of the Mississippi Teacher Performance Standards All data collection instruments and procedures are designed to collect information about these definition items, indicators, and standards. There are three data collection procedures: 1. A structured interview 2. At least two classroom observations 3. The evaluator’s review of documents/artifacts prepared by the teacher Not all of these instruments and procedures are used to gather data for every performance indicator, usually only one or two. The information is then synthesized to create an indicator score (a 1, 2, 3, or 4) using a four-point scale. Individual definition items are not always scored, but information related to them is combined to create the indicator scores, and indicator scores are then “rolled up” to create scores for the standards. The remainder of this manual contains detailed information on the appraisal procedures, processes, and instrumentation. First in the manual is a chart showing which data sources (instruments/procedures) are used to collect data pertinent to the indicators, followed by a copy of the teacher performance standards, indicators and definition items. Following that chart are explanatory examples of each data collection instrument and procedures for using each one. Finally, there is a detailed description of the scoring system and the resulting appraisal summary report. 1 MISSISSIPPI TEACHER APPRAISAL DATA SOURCES Standards and Indicators OB = Classroom Observation IN = Interview DOC = Document/Artifact Review A. Plans Instruction Consistent with State Standards to Facilitate High Levels of Learning for All Students OB A1. Plans units of instruction that are aligned with state standards and curriculum frameworks A2. Plans instruction to address individual differences in learners B. Delivers Instruction that Facilitates High Levels of Learning for All Students X X OB IN DOC IN DOC OB IN DOC X X X B2. Actively engages students in the learning process X B3. Uses principles of effective teaching X OB C1. Maintains an environment and climate conducive to learning X C2. Maximizes time available for instruction X C3. Manages student behavior to provide productive learning opportunities for all X D. Demonstrates Knowledge and Skill in Assessing/Facilitating Student Learning D1. Plans and implements systematic assessments D2. Considers state content standards and assessments when planning classroom assessments X D3. Uses classroom assessment results X X D4. Produces evidence of student academic growth under his/her instruction E. Communicates Effectively With Students, Peers, and Parents/Guardians DOC X B1. Demonstrates deep knowledge of content C. Organizes and Manages the Learning Environment and Maintains Student Behavior that is Conducive to Learning IN OB IN E1. Speaks clearly, correctly, and coherently X X E2. Writes clearly, correctly, and coherently X DOC X X E3. Establishes effective communication processes F. Committed to Continual Professional Growth and School Improvement OB IN DOC F1. Engages in continuous professional development X F2. Takes an active role in school improvement X * THE INSTRUMENTS MARKED WITH X (I.E., OBSERVATION, INTERVIEW, AND DOCUMENT/ARTIFACT) CONTAIN INFORMATION FOR ADDRESSING THE DESIGNATED INDICATORS. 2 MISSISSIPPI TEACHER PERFORMANCE STANDARDS Standard A. An effective teacher plans instruction consistent with state standards to facilitate high levels of learning for all students. Indicator A1. Plans units of instruction that are aligned with state standards and curriculum framework Definition Items develops/selects long-range goals and measurable, short-term objectives consistent with state standards and curriculum frameworks selects content that leads to accomplishment of goals and objectives selects instructional resources and activities that enables students to accomplish objectives maintains records of what has been taught and uses these data in further planning Indicator A2. Plans instruction to address individual differences in learners Definition Items identifies individual differences among students within a class (achievement, ability, interests, motivation, learning styles) addresses differences by varying content addresses differences by varying materials addresses differences by varying questions addresses varying learning styles of students Standard B. An effective teacher delivers instruction that facilitates high levels of learning for all students Indicator B1. Demonstrates deep knowledge of content Definition Items presents accurate and up-to-date information presents developmentally appropriate content relates content to life experiences presents content in logical pattern/sequence relates content to other subject areas uses relevant examples identifies misconceptions/lack of understanding presents lesson/content in alternative ways responds knowledgeably/accurately to questions 3 Indicator B2. Actively engages students in the learning process Definition Items communicates with students at their levels of language and comprehension encourages participation ensures equitable participation elicits responses encourages student sharing uses student ideas/responses seeks elaboration refers to other students engages students in knowledge/hypotheses varies roles in instructional process develops/selects activities that require student engagement stimulates and encourages higher-order thinking at appropriate developmental levels Indicator B3. Uses Principles of Effective Teaching Definition Items makes sure that students know what they are doing and why they are doing it relates lesson to objectives/emphasizes main idea sequences lesson establishes relationships adjusts lesson, when appropriate provides guided practice provides independent practice provides review/summarization uses variety of resources/materials and technology uses multiple representations and explanations promotes higher-order thinking Standard C. An effective teacher organizes and manages the learning environment and maintains student behavior that is conducive to learning. Indicator C1. Maintains an environment and climate conducive to learning Definition Items organizes available space, materials, and equipment to facilitate learning creates and maintains a colorful, inviting classroom or other environment accommodates variety of activities displays acceptance, appreciation, and encouragement of all students acknowledges and celebrates student successes 4 establishes time lines for completion of tasks/assignments establishes standards for academic performance holds students accountable/encourages quality indicates confidence uses verbal/nonverbal enthusiasm uses positive verbal language/nonverbal cues demonstrates respect does not use ridicule/sarcasm/personal criticism Indicator C2. Maximizes time available for instruction Definition Items begins promptly manages time allotted to activities and adjusts time when necessary minimizes loss of instructional time establishes routines and transitions and manages them efficiently discourages/redirects digressions minimizes teacher digressions minimizes student down time handles interruptions Indicator C3. Manages student behavior to provide productive learning opportunities for all Definition Items establishes, emphasizes, and monitors rules/procedures monitors the classroom and makes necessary adjustments to prevent undesirable behavior addresses behavior problems when they occur keeps students on task anticipates and intervenes uses verbal/nonverbal actions uses reasonable sanctions rewards appropriate behavior treats students fairly and equitably Standard D. An effective teacher demonstrates knowledge and skill in assessing and facilitating student learning. Indicator D1. Plans and implements systematic assessments Definition Items develops an assessment plan when planning an instructional unit uses a variety of assessment methods, including appropriate pre- and postassessments and informal collection of data during instruction 5 aligns assessments with unit objectives, content, and instruction analyzes results of assessments and uses these results when planning instruction monitors student learning and activities provides feedback re-teaches when necessary Indicator D2. Considers state content standards and assessments when planning classroom assessments Definition Items aligns classroom assessments with state content standards and curriculum frameworks aligns classroom assessments with content and format of state assessments in appropriate ways Indicator D3. Uses classroom assessment results Definition Items uses results to determine student progress and learning uses results to improve curriculum and instruction Indicator D4. Produces evidence of student academic growth under his/her instruction Definition Items provides assessment data and/or student work that show accomplishment of desired objectives by most students as a result of instruction provides assessment data and/or student work that show substantial learning gains by most, if not all, students as a result of instruction Standard E. An effective teacher communicates effectively with students, peers, and parents/guardians. Indicator E1. Speaks clearly, correctly, and coherently Definition Items uses standard speech pronounces words correctly uses vocabulary and style appropriate to level of audience speaks fluently without hesitations adjusts rate, volume, pitch when necessary organizes presentation 6 Indicator E2. Writes clearly, correctly, and coherently Definition Items spells words correctly uses correct grammar and mechanics writes legibly uses vocabulary and style appropriate to level of audience organizes written information Indicator E3. Establishes effective communication processes Definition Items develops and maintains systematic, two-way communication with students, school personnel, and parents/guardians uses a variety of methods to keep parents/guardians informed of curriculum focus and instructional activities and progress of students provides student assessment results to those who can benefit from them (e.g., students, parents, other faculty) seeks to involve parents/guardians in their children’s learning and in the school demonstrates respect and consideration for all those with whom he/she interacts Standard F. An effective teacher is committed to continual professional growth and school improvement. Indicator F1. Engages in continuous professional development Definition Items establishes a systematic program to improve weaker areas of performance takes formal coursework to enhance knowledge and skills, if appropriate participates in school, system, and state-sponsored professional development programs uses ideas from books, professional journals, professional organizations, and technological resources to improve instruction and learning applies knowledge gained from these experiences to the classroom for the benefit of students Indicator F2. Takes an active role in school improvement Definition Items identifies and assists in removing barriers to student learning initiates activities and projects in the classroom and school that are consistent with the school improvement plan works collaboratively with administrators, faculty, and staff to implement the school improvement plan 7 STRUCTURED INTERVIEW The structured interview is a discussion between the evaluator and the teacher about particular areas of the teacher’s knowledge, skills, and performance pertaining to the Mississippi standards for teachers. The team member asks the questions and records (scripts) the teacher’s responses. The teacher may bring illustrative supporting materials to this structured interview. Each structured interview question covers specific indicators found in the standards. However, the teacher’s response may contain information pertinent to more than one indicator. The questions emphasize why the teacher does something or how a teacher makes decisions for determining what to do. The interview data is to be considered together with other available data in developing appraisal scores. Question 1: Standard A, Indicator A1: Plans Units of Instruction that are Aligned with State Standards and Curriculum This question has two parts and addresses how you determine what students should know or be able to do at the end of a unit of instruction. You may include materials or documents in your discussion to assist in your explanation for this question and all other questions. a) How do you decide on your goals, objectives, and instructional strategies? b) How do you align what you plan to teach with state and local standards, curriculum frameworks, and results of state assessments? Question 2: Standard A, Indicator A2: Plans Instruction to Address Individual Differences in Learners This question has two parts and addresses how you plan instruction to address individual differences among students. a) b) How do you determine differences among students? How do you accommodate these differences? Question 3: Standard D, Indicator D1: Plans and Implements Systematic Assessments Standard D, Indicator D2: Considers State Content Standards and Assessments when Planning Classroom Assessments This question has two parts and addresses how you assess students’ performance and measure learner outcomes. a) Please describe your assessment planning and strategies, including the types of assessments you use and your reasons for using them. b) How do you align your classroom assessments with state content standards, curriculum frameworks, and state assessments? 8 Question 4: Standard D, Indicator D3: Uses Classroom Assessment Results This question has two parts and addresses what you do with your classroom assessment results. a) How do you use assessment results in determining student progress? b) How do you use assessment results in decisions about curriculum and instructional content? Question 5: Standard E, Indicator E3: Establishes Effective Communication Processes This question has two parts and addresses how you communicate with students, parents, and others. a) What do you communicate to students, parents, and others, how do you communicate it, and when do you communicate it? b) How you seek to involve parents/guardians in their children’s learning? Standard E, Indicator E1: Speaks Clearly, Correctly, and Coherently. There is no question for this indicator. The score is based on the interview delivery and organization. Illustrative Materials The purpose of providing illustrative materials is to enhance the interview through resources the teacher already has used. Teachers should not spend time creating new or different materials. Neither is the teacher expected to bring a large volume of items. In the interview, the materials should provide a representative sampling of what a teacher does in light of the established standards. Some materials may also be appropriate for more than one question if these materials illustrate different points. A teacher is not required to bring materials to the interview, but they can be useful to both the evaluator and the teacher. Recording the Interview Responses This manual contains the Mississippi Teacher Interview Scoring Form used for scoring the teacher’s responses to the interview questions. A Mississippi Teacher Interview Scoring Form has been provided for each interview question set. The form contains spaces for recording the teacher's name and school, the evaluator’s name, and the date of the interview. It also contains the interview questions (to be asked verbatim by the evaluator) with designation of the standard and indicator to which the question applies and a set of statements that will be scored after the interview is conducted. These statements are topics/issues that should be covered in the teacher's response. The statement list is also used by the evaluator to keep track of what has and has not been covered and to assist the evaluator in determining when probing questions are needed or when to move on to the next question. A separate form is provided for scripting or recording the teacher’s response. The evaluator will conduct the interview by asking the questions on the interview form, asking probing questions, and taking anecdotal notes on the scripting form. The evaluator will record what the teacher says and the probing questions asked by the evaluator. While the evaluator won't be able to record every word spoken, he/she will record as much of the dialogue as possible in order to determine later what has been said that depicts the teacher’s practices (how) 9 and rationale for those practices (why). Three rules for scripting interviews are essential: 1. The script should be complete enough that a person other than the evaluator can read it, even a year or two later, and describe what was said. 2. The relationship between the script and the score awarded for the indicator should be evident to a reader trained to analyze data of these kinds. 3. There should be no judgmental statements (no evaluator opinions) in the script—only a record of what was said. The evaluator has the responsibility of probing for more in-depth information throughout the discussion. Even though the evaluator may need to interrupt the teacher for clarification or more specificity, doing so is essential because the evaluator needs enough information to determine sufficiency of response. Thus, probing questions are necessary in order to get the information required for making that determination. In addition to the questions, each form contains scoring statements that will assist the evaluator with probing questions or determining when to move on to the next question. The evaluator must keep the interview moving. At the onset, the evaluator will make clear that discussion of any one question set should require no more than 7 to 8 minutes, that he/she will be watching the time, and that he/she will inform the teacher when it is time to “wrap up” and “move on.” 10 MISSISSIPPI TEACHER INTERVIEW SCORING FORM Teacher: ____________________________________________ School: _________________________ Evaluator: __________________________________________ Date: __________________________ Directions: This form is used for scoring the teacher’s responses to the interview questions. Attach scripting. Question 1 (Indicator A1) These two questions refer to how you determine what students should know or be able to do at the end of a unit of instruction. Please share any materials that will help you in your explanation for these and any following questions. a) How do you decide on your goals, objectives, and instructional strategies? 1. Long-range goals and measurable objectives 1 2 3 4 2. Planning that is aligned with state and local 1 2 3 4 1 2 3 4 1 2 3 4 standards and curriculum frameworks 3. Strategies and activities linked to goals and objectives b) How do you align what you plan to teach with - state and local standards, - curriculum frameworks, and - results of state assessments? Rationale: Indicator A1 Score: 11 MISSISSIPPI TEACHER INTERVIEW SCORING FORM Teacher: ____________________________________________ School: _________________________ Evaluator: __________________________________________ Date: __________________________ Directions: This form is used for scoring the teacher’s responses to the interview questions. Attach scripting. Question 2 (Indicator A2) These two questions refer to how you plan instruction to address individual differences among students. a) 1. Identifies differences influencing learning 1 2 3 4 2. Varies objectives, content, resources, activities and/or assessments for particular students 1 2 3 4 1 2 3 4 How do you determine differences among students? b) How do you accommodate these differences Rationale: Indicator A2 Score: 12 MISSISSIPPI TEACHER INTERVIEW SCORING FORM Teacher: ____________________________________________ School: _________________________ Evaluator: __________________________________________ Date: __________________________ Directions: This form is used for scoring the teacher’s responses to the interview questions. Attach scripting. (Question 3 (Indicator D1 and D2) These two questions address how you assess students’ performance and measure outcomes. a) Please describe your assessment planning and strategies, - including the types of assessments you use and why. b) How do you align your classroom assessments with -state content standards, -curriculum frameworks, and -state assessments? 1. Plans and carries out multiple assessments over the course of a unit 1 2 3 4 2. Uses formal and informal assessments 1 2 3 4 3. Uses a variety of assessments 1 2 3 4 4. Measurement linked to outcomes 1 2 3 4 Indicator D1 Score: 1 2 3 4 1. Alignment of assessment with state content standards 1 2 3 4 2. Analysis of state assessments, if applicable 1 2 3 4 1 2 3 4 Rationale: Indicator D2 Score: 13 MISSISSIPPI TEACHER INTERVIEW SCORING FORM Teacher: ____________________________________________ School: _________________________ Evaluator: __________________________________________ Date: __________________________ Directions: This form is used for scoring the teacher’s responses to the interview questions. Attach scripting. Question 4 (Indicator D3) These two questions address what you do with your classroom assessments. a) How do you use assessment results in determining student progress? 1. Use of assessment results to determine student progress, strengths, and areas needing improvement 1 2 3 4 2. Use of assessment results to improve curriculum and instruction 1 2 3 4 1 2 3 4 b) How do you use assessment results in decisions about curriculum and instructional content? Rationale: Indicator D3 Score: 14 MISSISSIPPI TEACHER INTERVIEW SCORING FORM Teacher: ____________________________________________ School: _________________________ Evaluator: __________________________________________ Date: __________________________ Directions: This form is used for scoring the teacher’s responses to the interview questions. Attach scripting. Question 5 (Indicator E3) These two questions address how you communicate with students, parents, and others a) What do you communicate to students, parents, and others; -how do you communicate it; and -when do you communicate it? 1. Established communication systems with parents, students, school personnel 1 2 3 4 2. Multiple types of information given to students, parents/guardians, and others 1 2 3 4 3. Involvement of parents/guardians 1 2 3 4 Indicator E3 Score: 1 2 3 4 b) How do you seek to involve parents/guardians in their children’s learning? Rationale: 15 MISSISSIPPI TEACHER INTERVIEW SCORING FORM Teacher: ____________________________________________ School: _________________________ Evaluator: __________________________________________ Date: __________________________ Directions: This form is used for scoring the teacher’s responses to the interview questions. (STANDARD INDICATOR E1) (There is no question. Scores are based on interview delivery and organization.) 1. Correct pronunciation 1 2 3 4 2. Appropriate vocabulary 1 2 3 4 3. Fluency 1 2 3 4 1 Indicator E1 Score: 1 2 2 3 3 4 4 4. Organization of presentation Rationale: Notes: 16 MISSISSIPPI TEACHER INTERVIEW SCRIPTING FORM Teacher: _____________________________ Evaluator: ______________________ Page: ____ of ____ Line Number 1. __________________________________________________________________________________________ 2. __________________________________________________________________________________________ 3. __________________________________________________________________________________________ 4. __________________________________________________________________________________________ 5. __________________________________________________________________________________________ 6. __________________________________________________________________________________________ 7. __________________________________________________________________________________________ 8. __________________________________________________________________________________________ 9. __________________________________________________________________________________________ 10. __________________________________________________________________________________________ 11. __________________________________________________________________________________________ 12. __________________________________________________________________________________________ 13. __________________________________________________________________________________________ 14. __________________________________________________________________________________________ 15. __________________________________________________________________________________________ 16. __________________________________________________________________________________________ 17. __________________________________________________________________________________________ 18. __________________________________________________________________________________________ 19. __________________________________________________________________________________________ 20. __________________________________________________________________________________________ 21. __________________________________________________________________________________________ 22. __________________________________________________________________________________________ 23. __________________________________________________________________________________________ 24. __________________________________________________________________________________________ 25. __________________________________________________________________________________________ \ 17 Scoring Interview Responses Information used to develop a score for each question set (indicator) is already organized by question in the script. However, it is possible that the teacher unknowingly provided information pertinent to one question when answering another. There are rules for dealing with this situation. The same information may not to be used to develop scores for two indicators. If it were, that would be double scoring. When information pertinent to a particular question was provided by a teacher in response to a different question, the evaluator considers that information in scoring that pertinent question. In other words, the evaluator is to consider information located anywhere in the interview when developing an indicator score. The Mississippi Teacher Interview Scripting and Scoring Form is used to record scores and rationales for the scores. To develop the scores and rationales, the evaluator follows a four-step process: 1. The evaluator analyzes the information provided in response to the question set and any other pertinent information “misplaced” in responses to other questions. 2. Using the 4-point scale below, the evaluator develops a score for each scoring statement on the scoring form. 4 - Exemplary Indicates that the teacher’s performance is exemplary in this position. The teacher almost always exceeds expected standards. 3 - Satisfactory Indicates that the teacher’s performance usually meets or exceeds expected standards. Areas noted can be improved, but current performance is clearly acceptable. 2 - Needs Improvement Indicates that the teacher’s performance is sometimes but not always acceptable in this position. The teacher sometimes meets expected standards. Improvement activities are necessary to meet standards on a consistent basis. 1 - Unsatisfactory Indicates that the teacher’s performance is not acceptable. Performance in this position rarely meets standards. 18 To make the score determination, the evaluator places a hypothetical mid-point line between the scores of 2 and 3. Then he/she will ask questions: a) Do the patterns in the data indicate a score above the mid-line or below it? b) If the answer to the first question is above the line, then the evaluator asks, “How far above the line is the pattern of evidence? Does it best represent a 4 or a 3?” If the answer to the first question is below the line, then the evaluator asks, “How far below the line is the pattern of evidence? Does it best represent a 1or a 2?” 3. The evaluator “rolls up” the scores from the scoring statements to create an indicator score. The evaluator does not average the scores, but uses pattern analysis. (What score do the pattern of scoring statement scores and the script information suggest?) 4. The evaluator writes the rationale (justification) for the score in the space provided. The rationale statement shows why the score was awarded by describing strengths and weaknesses in the information provided by the teacher in relation to the performance indicator. The evaluator refers to the definition items and key concepts in the process of writing the rationale. 19 THIS PAGE INTENTIONALLY LEFT BLANK 20 Key Concepts Key concepts used for probing questions and scoring the interview are provided below. Question 1: Standard A, Indicator A1: Plans Units of Instruction that are Aligned with State Standards and Curriculum This question has two parts and addresses how you determine what students should know or be able to do at the end of a unit of instruction. You may include materials or documents in your discussion to assist in your explanation of these and any following questions. a) How do you decide on your goals, objectives, and instructional strategies? b) How do you align what you plan to teach with state and local standards, curriculum frameworks, and results of state assessments? The focus of this question is on how the teacher makes decisions about what to teach. State and local curriculum frameworks, content research, professional publications, past appraisals, and other sources should be used in preparing goals and objectives. The teacher should show that the needs of learners as well as a relationship to the curriculum are addressed. Discussion should focus on outcomes or what is expected of students. 1. Long-range goals and measurable objectives Effective teachers plan on more than a day-to-day basis. They establish goals for a unit of work and objectives that define those goals and specific outcomes desired as a result of instruction. Sources for goals and objectives are state standards and curriculum frameworks, local curriculum plans, national content standards, and professional literature. 2. Planning that is aligned with state and local standards and curriculum frameworks Effective teachers match their goals, objectives, and selection of content to state standards and local curriculum frameworks. 3. Strategies and activities linked to goals and objectives Effective teachers select strategies and resources that are directly aligned with objectives and assessments, including state assessments. Question 2: Standard A, Indicator A2: Plans Instruction to Address Individual Differences in Learners This question has two parts and addresses how you plan instruction to address individual differences among students. a) How do you determine differences among students? b) How do you accommodate these differences? The teacher should discuss strategies used to identify individual differences in students. Consideration should be given to such differences as student achievement, differences in abilities, varying interests, special needs, and learning styles. Because of differences among students, teachers should plan a variety of objectives, activities, and assessments. Effective teachers also have a variety of ways to identify and accommodate differences among students. 21 1. Identifies differences that influence learning Effective teachers identify differences among students in a given class that must be addressed. Such differences include development, ability, achievement, interests, motivation, learning styles, ethnicity, culture, and language. The teacher should be able to discuss how he/she goes about identifying differences and which ones have regularly emerged. 2. Varies objectives, content, resources, activities and/or assessments for particular students The teacher should be able to explain how he/she has addressed differences identified in one or more of these ways. Examples of accommodations made for specific students should be requested. Question 3: Standard D, Indicator D1: Plans and Implements Systematic Assessments Standard D, Indicator D2: Considers State Content Standards and Assessments when Planning Classroom Assessments This question has two parts and addresses how you assess students’ performance and measure learner outcomes. a) Please describe your assessment planning and strategies, including the types of assessments you use and why. b) How do you align your assessments with state content standards, curriculum frameworks, and state assessments? The effective teacher knows the accuracy and limitations of various assessments. The teacher should use a variety of assessments, including paper-and-pencil tests where appropriate, and a variety of performance assessments (portfolios, observation, performance tasks) to create a comprehensive formal assessment program. Teacher monitoring of student progress during a lesson is one type of informal assessment. Assessments should address and be aligned with unit objectives. The teacher should use assessment results to determine the success of instruction, activities, or resources. Assessments should demonstrate a relationship to state and local content standards, curriculum frameworks, and state assessments, particularly in format. Indicator D1: Plans and Implements Systematic Assessments 1. Plans and carries out multiple assessments over the course of a unit Effective teachers plan their programs and patterns of assessment when they plan an instructional unit. 2. Use of formal and informal assessments Effective teachers understand that assessment methods should be both formal and informal. Teachers assess students over the course of a unit in more than one way. 3. Uses a variety of assessments Effective teachers use more than paper-and-pencil tests or monitoring activities. Portfolios, oral testing, projects, observation, and performance tasks provide a variety of assessments. 22 4. Measurement linked to outcomes Teachers look at expected student outcomes and select or design assessments to determine the success of those outcomes. Indicator D2: Considers State Content Standards and Assessments when Planning Classroom Assessments 1. Alignment of assessment of state content standards The teacher should be able to describe how he/she aligns assessments with the student behaviors and practices identified in state content standards. 2. Analysis of state assessments The teacher should be able to explain how he/she has used the content and format of state assessments in planning classroom assessments. Question 4: Standard D, Indicator D3: Uses Classroom Assessment Results This question has two parts and addresses what you do with your classroom assessment results. a) How do you use assessment results in determining student progress? b) How do you use assessment results in decisions about curriculum and instructional content? Teacher discussion should focus on how he/she uses the results of assessment. How the results are reported should be discussed in Question 5. The teacher may use results to determine student strengths and needs, to determine adjustments in content or instruction, and to determine the success of instruction. 1. Use of assessment results to determine student progress, strengths, and areas needing improvement The effective teacher uses assessment results to determine student accomplishment of objectives and gains in knowledge and skills. The teacher also uses assessments to identify areas of strength and need and to understand how these factors affect a student’s progress. The teacher should have a system of record keeping for ensuring the accuracy of student progress and indicating areas of strength or needing improvement. 2. Use of assessment results to improve curriculum and instruction Effective teachers use assessment results to determine the success of instruction and needed adjustments to units/lessons. Considerations should be given to appropriateness of objectives, teaching strategies, content, student activities, resources, and the assessments used. Question 5: Standard E, Indicator E3: Establishes Effective Communication Processes This question has two parts and addresses how you communicate with students, parents, and others. a) What do you communicate to students, parents, and others, and how do you communicate it, and when do you communicate it? b) How do you seek to involve parents/guardians in their children’s learning? 23 The teacher should describe how he/she provides feedback to students about their progress, strengths, and areas of need. Descriptions of how feedback is provided to parents and appropriate others should also be discussed. In addition to the typical report card, the teacher may use such things as conferences with students and parents, written feedback, the Internet, and progress charts. The discussion should also include what is communicated to students, parents, and others. Time lines and specificity should be considered in providing feedback. 1. Established communication systems with parents, students, school personnel Effective teachers have in place a system of communication for providing relevant information to parents, students, and school personnel. The teacher should use a variety of procedures in reporting assessment results to students, parents, and appropriate others (principal, other teachers). Letters, phone calls, progress reports and charts, and in-class evaluation sessions provide several sources of feedback. Information is provided in a timely manner. There should be two-way communication. 2. Multiple types of information given to students and parents/guardians Students should be provided with information about what is expected of them and with feedback on how well they are progressing. Parents/caregivers are contacted concerning student progress, subject matter, forthcoming events, standards, discipline expectations, school and instructional goals and objectives, content of the curriculum, and expectations of the home. Multiple types of information also include multiple approaches for presenting the same information (written explanation and graphical representation, web-based delivery, and reports sent home). 3. Involvement of parents/guardians Effective teachers invite and provide opportunities for parents/guardians to participate in the education of their children. Parents can assist teachers and students in the classroom and the computer lab, on school outings, during special projects and activities, and with students who have specific individual needs. Parents can also be enlisted to work with students on particular skills. Standard E, Indicator E1: Speaks Clearly, Correctly, and Coherently There is no question for this indicator. The score is based on the interview delivery and organization. 1. Correct pronunciation The teacher consistently speaks distinctly, avoids mumbling or slurring of words, and pronounces words correctly. 2. Appropriate vocabulary The teacher uses vocabulary that can be understood and that demonstrate command of the language. 24 3. Fluency The teacher avoids excessive use of repetition of words or ideas. The teacher should also avoid the use of “uhs”, “ahs”, “you know,” and vague words or terms. 4. Organization of presentation The teacher gives evidence of organized presentation by avoiding gaps in the presentation or periods when he/she appears to search for the next point or statement. There is logical, sequential presentation of ideas, rather than rambling. Ground Rule: A score of “3” is considered to be the norm and what is expected. Award a “4” only if the teacher has patterns of behavior that demonstrate he/she communicates at a level above what is expected. Award a “2” or lower if there are patterns of errors. Specific examples should be included in the rationale statement for a score of 2 or less. 25 THIS PAGE INTENTIONALLY LEFT BLANK 26 CLASSROOM OBSERVATION Observation is very complex. It is no easy task to observe all that can possibly be observed and record significant practices and behaviors. When observing human interactions, it is necessary to watch and listen to receivers of communication as well as senders. Therefore, it is necessary to train evaluators to recognize discreet verbal and nonverbal behaviors, the presence or absence of artifacts in the environment, and the presence or absence of patterns of behavior that indicate well-established, effective routines and procedures. The observation process is made even more complex by the matter of style. Studies of effective teaching have established that good teachers vary in their styles, but deliver very similar behaviors and practices, which produce higher achievement in learners. In other words, we know that good teachers look quite different from one another. They do the same kinds of things, but they don’t all do them in the same way. For example, effective teachers (those who produce higher student achievement) communicate the purpose(s) of each lesson to students, but they vary in regard to approach, explicitness, and point in the lesson where this communication takes place. Most people would give lip service to the notion that good teachers differ in their styles. However, those people might have subconscious stereotypes of an effective teacher. Failure to recognize differences in teaching styles can produce two types of errors in the appraisal process: (1) failure to credit a teacher for an indicator or standard that is present and (2) failure to recognize the lack of a specific indicator or standard. The training for evaluators in the Mississippi Teacher Appraisal is designed to break down stereotypes, develop sensitivities to various styles, and sharpen the ability to recognize effective behaviors and practices. Many, if not all, of the standards, indicators, and definition items that form the appraisal criteria for teachers require the same expertise on the part of the evaluator. One recognizes practices and behaviors that demonstrate the indicator or item, but one must not expect a single set of practices or behaviors across teachers. Awareness of differences in teaching style and practices helps to make evaluators more objective in their observations. Training heightens that awareness and cause evaluators to focus on effectiveness and not their own images of good teaching. The evaluator is basically just that—an evaluator of events, processes and procedures. From the lowinference information about events, processes, and procedures observed, he/she identifies patterns of behavior and practice that demonstrate the teacher’s competence regardless of style. Observation Approach and Requirements The Mississippi Teacher Appraisal System uses the scripted observation approach. In this approach, the evaluator writes down, either as direct quotes or narrative explanations, whatever is seen or heard that pertains to the appraisal criteria. 27 Two observations should be conducted for each teacher in the school. Both observations are to be announced observations. The second observation is to be conducted by a different evaluator than the first. In some cases, the evaluator(s) may conduct more than two observations if it is determined that more information is needed. In these cases, the additional observation(s) can be announced or unannounced and may be conducted by a third evaluator. Mississippi Teacher Classroom Observation Record In using the Mississippi Teacher Classroom Observation Record, the evaluator carries out two functions. First, he/she records what the teacher and students say and do that is relevant to the performance indicators under investigation. These comments and direct quotes are written into the record without judgment. The evaluator’s second function is to analyze and synthesize the data recorded to guide discussion with the teacher in the appraisal conference as well as to identify areas for future professional growth. The evaluator also uses information from the analysis and synthesis of the observation behaviors to determine indicator scores. These summary ratings are recorded on the last page of the record and include comments (rationales) supporting the scores. The 4-point scale discussed in the interview section of this manual is used for scoring. In addition to conducting the observation and analyzing the information generated, the evaluator is required to collect pre-observation information for announced observations from the teacher being observed. Together these activities (pre-conference and observation) constitute the observational component of the Mississippi Teacher Appraisal. Pre-Observation Conference A Mississippi Teacher Pre-Observation Conference Record is part of the package of instruments used in the classroom observation. Before the conference, the evaluator can complete some of the demographic information at the top of the Mississippi Teacher PreObservation Conference Record. The information recorded here is not scored. It is simply used to prepare the evaluator for the observation. Following the discussion, both parties sign the Mississippi Pre-Observation Conference Record as evidence that the pre-observation conference was held. The pre-observation conference is an important part of the sequence of announced observations. During this conference, the evaluator establishes a perspective on the setting, situation, or events to be observed, as well as the participants in the environment. This conference period also provides information about conditions that may impact the teacher's performance. Further, and perhaps more important, the conference provides an opportunity for the evaluator to establish greater rapport with the teacher. Rapport is essential to a “helping” relationship, to the post-observation sharing of strengths and areas for improvement, and to proposing of recommendations for improvement. Pre-observation conferences are required for all announced observations. Announced observations are scheduled with the teacher a day or more ahead of time. Unannounced observations take place when an evaluator arrives at the classroom or instructional site without pre-scheduling and may occur if a third classroom observation is deemed necessary. It is best if a short pre-observation conference is arranged for these unannounced observations so 28 that the evaluator can understand what he/she will be seeing. However, this is sometimes impossible. When pre-observation conferences are included, about 15 minutes are allocated for them. For unannounced observations, 5 to 6 minutes may have to do. Some details on the form may have to be filled in at a later time. The pre-conference for announced observations takes place on the day of the observation or a day before, if the teacher and evaluator agree to that schedule. A teacher may be asked to complete the information on the pre-observation conference record prior to the conference. However, the evaluator and teacher still hold a conference so both parties know what will be happening in the classroom. The pre-observation conference is also the time to find out if any special conditions prevail in the classroom environment. There are places to mark special conditions that the teacher wants noted. These might include classroom conditions that could affect instruction, such as inadequate technology, inadequate facilities, inadequate resources, or “other” as defined by the teacher. The teacher is the one to identify special conditions. Later these conditions are considered, as appropriate, in the appraisal. The pre-observation conference questions focus on topics or content to be covered in the classroom observation, teacher and student activities, and how the accomplishment of objectives will be measured. The evaluator asks the teacher whether the instruction is new content or re-teaching. This question sometimes clears up some confusion for the evaluator once the lesson begins. The evaluator also reminds the teacher that he/she will be writing (scripting) the entire visit and that a seat where the evaluator can observe both the teacher and students is needed. The evaluator may also remind the teacher to give him/her copies of any handouts the children receive. No Post-Observation Conference There is no post-observation conference. However, during the pre-observation conference, the evaluator explains to the teacher that the results of the observation will be discussed during the final Appraisal Conference. Conducting the Observation The evaluator selects a seat where he/she can observe both the teacher and the students. The evaluator begins recording information when the lesson begins. He/she records continuously. In secondary classrooms, this pattern is maintained until the end of the class period. In secondary and middle school classrooms, observations are to be no less than 45 minutes in length (full class period). If subject matter changes during that time, the evaluator records that information in order to make sense of what might seem to be conflicting data when reviewed later. Some elementary classroom lessons are less than 45 minutes. Each elementary classroom observation is to include a minimum of 30 minutes of instruction. If instruction is shorter than that, two different visits are required to gather enough data to generate valid scores for the observation. If an elementary teacher shifts from one content area to another in less than 30 minutes, the evaluator simply notes the shift in order to make sense of what might seem to be conflicting data when reviewed later. 29 In classrooms using block scheduling or extended periods of instruction, the evaluator is not expected to observe the whole class period. In this situation, the evaluator observes the first 25 minutes, and then returns for the final 20 minutes. Scripting the Observation The Mississippi Teacher Classroom Observation Record is constructed to facilitate scripting a narrative description of what the evaluator sees and hears. The instrument uses the Mississippi Teacher Observation Scripting Form for recording these narrative descriptions. Several pages of the scripting form are needed to record what takes place over a 45 to 50 minute period. . The procedure used is commonly known as modified scripting. In this approach everything the evaluator sees and hears that applies to one or more of the standards and indicators being assessed is recorded. This rule implies that not everything that is done and said in the classroom needs to be recorded (full scripting), but that all teacher and student actions pertaining to standards/indicators are clearly visible at the end of the observation. If the evaluator has questions about the pertinence of a behavior when it occurs, time is not wasted trying to determine that pertinence—it is written down. It is better to record too much information than too little. Script pages contain places for the teacher’s and evaluator’s names. These pages also contain a box in the upper right corner to place a page number. The names and page numbers are important because they offer a reference point and sequence if pages somehow become separated. Script pages also contain line numbers. Line numbers are provided so that there will be a ready reference to specific behaviors or practices when completing the Mississippi Teacher Observation Analysis and Scoring Form. An evaluator can use page and line numbers to find material that should be shared with the teacher. A faint line runs vertically down the middle of the script pages. The line has been placed there to aid evaluators who wish to script teacher behavior on one half of the page and related student behavior on the other, or to script using two columns. Note: When scripting, the evaluator will stop three times during the observation and count the number of students “off task.” He/she will record this number in the script with the note “off task.” Classroom Observation Record (cover sheet) The Classroom Observation Record is a cover sheet for the observation process. It is to be completed after the observation. The evaluator completes the top section based on information obtained during the pre-observation conference. Based upon the actual observation, the evaluator states if special conditions were observed other than those previously indicated by the teacher and if the activities were those stated by the teacher. If not, he states the activities that were observed. 30 Mississippi Teacher Observation Analysis and Scoring Form In completing the Mississippi Teacher Observation Analysis and Scoring the evaluator considers the lesson as a whole. After the lesson and script are completed, the evaluator needs time to analyze and synthesize the data around the indicators. These analyses can be done in several ways. For example, the evaluator might commonly number or label all entries in the script that pertain to a specific indicator. Or, he/she might create a list of those behaviors. Another method is to cite page and line numbers for sections of the script in which entries pertaining to an indicator are found. Whatever technique is used to identify related entries, supporting references for use in the evaluation discussion with the teacher will be needed. These references and the “raw” script data provide the bases for the discussion in the final appraisal conference. Space has been provided to note where supporting evidence is found in the script. Quality as well as quantity should be considered. Note: Conferring with the second evaluator concerning the initial observation is not permitted. Also, making any comments, positive or negative, that might influence his/her observation or scoring of the teacher is not permitted. If there is a significant discrepancy between the scores of the two observations, the team leader is to be informed so a decision concerning a third observation can be made. 31 MISSISSIPPI TEACHER PRE- OBSERVATION CONFERENCE RECORD Teacher: __________________________________________ Evaluator: _________________________________ School System: ________________________ School: ________________________ Date of Obs: ____/____/____ Subject/Grade Level: _________________________________ Number of Students: ________ Time: __________ Observation No. 1 2 Multiple Disorders Inadequate Facilities 3 Special Conditions (to be specified by teacher) Overcrowded Conditions Inadequate Resources Inadequate Technology Other Comments: ______________________________________________________________________________ 1. What topic(s), knowledge, or skills will be covered in this observation, and how does this lesson relate to previous lessons? What will be the DOK level? 2. What student and teacher activities are planned? 3. How and when will objectives be measured? Teacher-Made Tests System-Developed Tests Commercially Prepared Tests Textbook Tests Homework Work Products/Projects 4. Teacher Comments: (optional) 5. Evaluator Comments: (optional) Standardized Tests Student Performance Other ___________ Teacher Signature: ________________________________________________________ Date of Pre-Conference:_________________________ Evaluator Signature: ______________________________________________________ Date of Pre-Conference:_________________________ Teacher's signature indicates this form has been reviewed and a Pre-Observation Conference has been held. 32 MISSISSIPPI TEACHER CLASSROOM OBSERVATION RECORD Teacher: ____________________________________ Evaluator: _____________________________ School System: ________________________ School: ______________________ Observation No. 1 2 3 Grade/Subject: ________________ Date: ___/___/___ Time: __________ SPECIAL CONDITIONS (To be specified by teacher) Special Population Overcrowded Conditions Inadequate Technology Inadequate Facilities Inadequate Resources Subject Change Other ______________ Special conditions observed (other than those indicated by the teacher): Were student/teacher activities observed those planned? If no, indicate activities observed. Yes No Comments: ____________________________________________________________________________________ After scripting the observation, behaviors related to each indicator should be referenced on the Mississippi Teacher Observation Analysis and Scoring Form. The following scale is used to score the indicators. 4- Exemplary indicates that the teacher’s performance is exemplary in this position. The teacher almost always exceeds expected standards. 3- Satisfactory indicates that the teacher’s performance usually meets or exceeds expected standards. Areas noted can be improved, but current performance is clearly acceptable. 2- Needs Improvement indicates that the teacher’s performance is sometimes but not always acceptable in this position. The teacher sometimes meets expected standards. Improvement activities are necessary to meet standards on a consistent basis. 1- Unsatisfactory indicates that the teacher’s performance is not acceptable. Performance in this position rarely meets standards. 33 MISSISSIPPI TEACHER OBSERVATION ANALYSIS AND SCORING FORM Teacher: __________________________________ Evaluator: ____________________________ Plans Instruction Score: Rationale/Supporting Script References: ______ A2. Individual Differences Addresses differences by varying content Addresses differences by varying materials Addresses differences by varying questions Addresses varying learning styles Delivers Instruction Score: Rationale/Supporting Script References: ______ B1. Knowledge Presents accurate and up-to-date information Presents developmentally appropriate content Relates content to life experiences Presents content in logical pattern/sequence Relates content to other subject areas Uses relevant examples Identifies misconceptions/lack of understanding Presents lesson/content in alternative ways Responds knowledgeably/accurately to questions ______ B2. Engages Students Encourages participation Ensures equitable participation Elicits responses Encourages student sharing Uses student ideas/responses Seeks elaboration Refers to other students Engages students in knowledge/hypotheses Varies roles in instructional process ______ B3. Uses Principles of Effective Teaching Orients students to lesson Relates lesson to objectives/emphasizes main idea Sequences lesson Establishes relationships Adjusts lesson, when appropriate Provides guided practice Provides independent practice Provides review/summarization Uses variety of resources/materials and technology Uses multiple representations and explanations Promotes higher-order thinking 34 Learning Environment Teacher : __________________________________ Score: Evaluator: ____________________________ Rationale/Supporting Script References: ______ C1. Maintains Climate Establishes timelines for completion Establishes standards Holds students accountable/encourages quality Indicates confidence Displays verbal/nonverbal enthusiasm Uses positive verbal language/nonverbal cues Demonstrates respect Avoids use of ridicule/sarcasm/personal criticism Accommodates variety of activities Maintains attractive environment ______ C2. Maximizes Time Begins promptly Minimizes loss of instructional time Establishes routines Discourages/redirects digressions Minimizes teacher digressions Minimizes student down time Handles interruptions ______ C3. Manages Student Behavior Emphasizes and monitors rules/procedures Keeps students on task Anticipates and intervenes Uses verbal/nonverbal actions Uses reasonable sanctions Rewards appropriate behavior Treats students fairly and equitably Assessment Score: Rationale/Supporting Script References: ______ D1. Implements Assessment Monitors student learning and activities Provides feedback Re-teaches when necessary Communication Score: Rationale/Supporting Script References: ______ E1. Speaks Clearly, Correctly, Coherently Adjusts rate, volume, pitch Speaks fluently/uses correct pronunciations Organizes presentation ______ E2. Writes Clearly, Correctly, Coherently Uses correct grammar/mechanics/spelling Writes legibly Organizes information Number of students in class: _______ Number of students off-task: ____ ____ ____ ____ 35 MISSISSIPPI TEACHER OBSERVATION SCRIPTING FORM Teacher: ______________________________ Evaluator: ______________________ Page: ____of_____ Line Number 1. __________________________________________________________________________________________ 2. __________________________________________________________________________________________ 3. __________________________________________________________________________________________ 4. __________________________________________________________________________________________ 5. __________________________________________________________________________________________ 6. __________________________________________________________________________________________ 7. __________________________________________________________________________________________ 8. __________________________________________________________________________________________ 9. __________________________________________________________________________________________ 10. __________________________________________________________________________________________ 11. __________________________________________________________________________________________ 12. __________________________________________________________________________________________ 13. __________________________________________________________________________________________ 14. __________________________________________________________________________________________ 15. __________________________________________________________________________________________ 16. __________________________________________________________________________________________ 17. __________________________________________________________________________________________ 18. __________________________________________________________________________________________ 19. __________________________________________________________________________________________ 20. __________________________________________________________________________________________ 21. __________________________________________________________________________________________ 22. __________________________________________________________________________________________ 23. __________________________________________________________________________________________ 24. __________________________________________________________________________________________ 25. __________________________________________________________________________________________ 26. __________________________________________________________________________________________ 36 DEFINITIONS AND EXPLANATIONS OF TEACHING PRACTICES How to Use the Definitions and Explanations The explanations and definitions found on the previous pages will be applied to the statements and quotes in the script when the evaluator analyzes the “raw data.” To do that, an evaluator must be thoroughly familiar with the indicators and definitions that make up that indicator. In addition, the evaluator must have an in-depth understanding of the instructional process so that when he/she sees and hears something, there is understanding of what the teaching function is and what it is not. When scripting and/or analyzing script entries, the evaluator will be asking the following series of questions: What is happening here? What do these statements, questions, actions represent – what aspect (function) of the teaching process? What indicator? What particular teacher behavior within this teaching function is taking place? What is missing? What teacher behaviors didn’t occur? Once script data have been organized on the Mississippi Teacher Observation Analysis and Scoring Form by one of the methods previously described, a score can be assigned to each indicator based on the strength or weakness of the teacher behaviors indicated. The four-point scale is used throughout the Mississippi Personnel Appraisals: 4 - Exemplary indicates that the teacher’s performance is exemplary in this position. The teacher almost always exceeds expected standards. 3 - Satisfactory indicates that the teacher’s performance usually meets or exceeds expected standards. Areas noted can be improved, but current performance is clearly acceptable. 2 - Needs Improvement indicates that the teacher’s performance is sometimes but not always acceptable in this position. The teacher sometimes meets expected standards. Improvement activities are necessary to meet standards on a consistent basis. 1 - Unsatisfactory indicates that the teacher’s performance is not acceptable. Performance in this position rarely meets standards. 37 Definitions and Explanations Indicator A2. Plans Instruction to Address Individual Differences in Learners 1. Addresses differences by varying content The effective teacher addresses student differences (achievement and ability levels, interests, learning styles) and plans instruction consistent with the developmental level of students (physical, social, emotional, and cognitive) by presenting content to meet those needs. 2. Addresses differences by varying materials The effective teacher addresses student differences (achievement and ability levels, interests, learning styles) and plans instruction consistent with the developmental level of students (physical, social, emotional, and cognitive) by providing materials that meet those needs. 3. Addresses differences by varying questions The effective teacher addresses student differences (achievement and ability levels, interests, learning styles) and plans instruction consistent with the developmental level of students (physical, social, emotional, and cognitive) by asking higher-order questions that challenge the abilities of students. Some students may be able to respond to synthesis or evaluation questions, while others may need questions on the application or analysis levels. The teacher also uses both divergent and convergent questions. 4. Addresses varying learning styles The effective teacher is aware of the various learning styles among the students in the classroom. Some students need an auditory presentation, others need print, and still others need to use their hands or participate physically in a learning activity. The teacher presents the lesson to address the learning styles of all students through the use of learning centers, displays, role-playing, and so forth. Note: Addressing differences means addressing individual differences - not doing the same thing, using the same materials, same style, etc. with the whole class. Indicator B1. Demonstrates deep knowledge of content 1. Presents accurate and up-to-date information When presenting a lesson or unit, the effective teacher presents correct and current information. The teacher clarifies or extends the lesson by drawing examples from current events. 2. Presents developmentally appropriate content The effective teacher selects content that is appropriate to the interests, maturation, and ability levels of students. 38 3. Relates content to life experiences The effective teacher relates content to the experiences of students. The teacher uses examples from life experiences (demonstrates relevance to everyday life or direct involvement of student or teacher) to help students relate the learning and understand concepts. 4. Presents content in logical pattern/sequence The teacher should order the presentation of the lesson. Content of the lesson or unit should include concepts and skills that follow or precede each other rationally. 5. Relates content to other subject areas The effective teacher relates the content of one lesson to other lessons. Concepts, principles, and skills taught are applied across curriculum areas. 6. Uses relevant examples The teacher provides examples or asks students to provide examples of content, topics, or principles. Examples provide clarification or extension of the lesson and may be such things as pictures, demonstrations by the teacher or students, objects, or media. 7. Identifies misconceptions/lack of understanding The teacher uses effective questioning to clarify misconceptions or lack of understanding by students. He/she requests students to verbalize their confusion and responds by modifying, prompting, or rephrasing to help them clear up any misconceptions. In group processes, the teacher may interview students by using questions dealing with problems hampering group or individual learning. At times, such questions assist groups in solving problems themselves. 8. Presents lesson/content in alternative ways The effective teacher does more than implement a direct teaching style when presenting a lesson. The teacher provides students with opportunities to have input into the lesson through discussion, projects, or presentations. Students may be taken outside the school to observe the concepts they are studying. They may even participate, such as when a group of students goes to a market to learn how employees carry out their tasks. Teachers may use technology such as computers to bring the outside world closer through the Internet. 9. Responds knowledgeably/accurately to questions The teacher responds to student questions by providing information that is current and correct. The teacher may also provide other sources to help students find answers to their questions or increase their knowledge. Indicator B2. Actively Engages Students in the Learning Process 1. Encourages participation One way that a teacher encourages student participation in the lesson is to call on non-volunteers. He/she structures questions, activities, and resources to 39 meet student differences and help all students become involved. For example, a teacher may structure the lesson so that certain students can succeed, thereby encouraging them to participate. A teacher takes individual interests and learning styles of students into consideration. The teacher also responds to students who have expressed an interest to respond to a particular lesson or activities. 2. Ensures equitable participation The teacher ensures that all students have equal opportunities to participate in the learning activities regardless of their abilities. The teacher may make adjustments to time or activities so that all students can participate. For example, the teacher may rotate students to allow each one to have the same amount of time on the computer, although activities on the computer may vary. 3. Elicits responses The teacher attempts to get a response from all students. A response could be to a question or to an activity. If students are not responding, he/she uses questioning, statements, recommendations, or paraphrasing to get the student to participate. 4. Encourages student sharing The teacher provides opportunities for students to work together. The teacher may have students work in small groups or in pairs to help each other. He/she may ask one student to help another or provide a task for pairs of students. For example, “Each student should choose a partner and work in pairs to come up with a list of reasons why people are dissatisfied with the economy.” 5. Uses student ideas/responses During the lesson, a teacher may use students’ comments to extend the lesson. Such student comments may help to further develop a concept or skill or illustrate a point related to the lesson. The teacher does more than just answer a student question. He/she actually makes use of the student’s comment. 6. Seeks elaboration The teacher requests that a student elaborate or clarify a previous response. Rather than attempting to get an initial response as in Elicits Response, the teacher asks a question or makes a statement that solicits additional information from the student about his/her response. 7. Refers to other students The teacher refers a student’s response to another student or students for elaboration or reaction. For example, a teacher may say, “Tom, do you agree with Nancy? Why or why not?” or “Anne, would you restate Bobby’s comments for us?” 40 8. Engages students in knowledge/hypotheses The teacher encourages students to learn how to learn by providing lesson activities or related discussions. Students are provided opportunities to search for information, develop questioning skills, or use several sources of information to draw conclusions. For example, in a government class studying city government, the teacher may have an activity where the students use articles from periodicals, internet sources, and books to develop models of different city operations (e.g., governing through council, mayoral vs. city manager approaches). 9. Varies roles in instructional process The teacher plays different roles within selected instructional strategies: presenter/explainer, facilitator/helper, coach, participant, etc. Indicator B3. Uses Principles of Effective Teaching 1. Orients students to lesson The effective teacher provides a clear description of the learning task. The teacher may orient students to the lesson in several ways, such as securing students’ attention, stating the purpose or objectives of the lesson, identifying content/skills to be mastered, and relating the lesson to previous/future lessons and to life experiences. 2. Relates lesson to objectives/emphasizes main idea The teacher relates the content of the lesson to the objectives. He/she also involves students in accomplishing the lesson objectives. Lesson activities are congruent with the objectives. 3. Sequences lesson The teacher orders the lesson by giving directions for independent tasks as they should occur. When more than one task is to be completed, methods to be used or directions for accomplishing multiple tasks are presented in sequence. 4. Establishes relationships The teacher helps students relate previously learned concepts, principles, or skills to new ones contained in the current lesson. The teacher may point out logical connections, relationships of facts, concepts, and skills to one another by showing similarities and/or contrasts, cause and effect, or whole/part relationships. 5. Adjusts lesson, when appropriate Through monitoring students, the teacher is aware of the need to adjust pacing, to present content that is easier or more challenging, or to use different materials or resources. 41 6. Provides guided practice The teacher provides teacher-led practice for the concepts and skills in the lesson. The practice is supervised and directed by the teacher, who also controls the pace of the lesson activity. The teacher provides immediate and frequent feedback to assess student understanding and to determine readiness to proceed to independent practice. 7. Provides independent practice The teacher provides opportunities for students to engage in independent work, either as homework or in class activities. The teacher may provide practice for individuals or groups and may ask questions and provide explanations during the independent activities. Homework can be considered independent practice. 8. Provides review/summarization The teacher may provide review at the beginning, middle, or end of a lesson, learning activity, or transition to a new aspect of the lesson. Review could also occur when summarizing or asking students to summarize or restate the current lesson or previous learning. The summary may include how students can continue their learning outside the classroom or a link may be made between the activities of the lesson and future or past lessons. Responding to a student question by reiterating previous work is not considered review. Nor are listing activities or discussion of what students have done previously considered a review. Review differs from re-teaching in that review is merely summarizing, clarifying, or highlighting previously learned material, while re-teaching is presenting content again in a step-by-step fashion using the same or different methods. Re-teaching and not review is based on a lack of understanding. 9. Uses variety of resources/materials and technology The effective teacher chooses resources appropriate to the content of the lesson as well as to the varying individual needs of students. For example, providing the same materials for all students would not be as appropriate as providing materials relevant to the particular advanced abilities or remedial needs of specific students. Some students have more experience with technology such as computers and others need beginning instruction. Look for a match among content, materials, and students. 10. Uses multiple representations and explanations The teacher uses a multisensory approach (hands-on, experience, experiments, charts, diagrams, real life situations) in developing the elements of the lesson. 11. Promotes higher-order thinking A teacher may stimulate higher-order thinking in students through questioning. The teacher may ask a student to compare or contrast two or more things or concepts. The teacher may ask students to apply cognitive skills of application, analysis, synthesis, or evaluation through discussion or questioning. 42 Indicator C1. Maintains an Environment and Climate Conducive to Learning 1. Establishes time lines for completion The teacher tells students when their projects, activities, or homework assignments are due. The teacher also holds students accountable for meeting those time lines. 2. Establishes standards Through discussion or teacher action, the teacher establishes or develops new standards with students. For example, a teacher may tell the class, “I expect you to check your papers for grammatical and spelling errors before you turn them in. Don’t forget to check for typos, also.” 3. Holds students accountable/encourages quality The teacher encourages responsibility among students for the work that they do. For example, a teacher may say, “Don’t forget that I expect all of you to work together by each one contributing to the completion of the project and not letting just a few do all the work.” 4. Indicates confidence The teacher promotes confidence that all students can learn or that they have the ability to complete or master the task at hand. A teacher may indicate confidence by saying, “During yesterday’s lesson, you discovered three different ways to solve the word problems. I’ll bet that you can find another one today.” 5. Displays verbal/nonverbal enthusiasm The teacher expresses interest and enthusiasm about the lesson or activity through positive comments, gestures, or facial expressions. The teacher may also provide humor during discussions or activities. 6. Uses positive verbal language/nonverbal cues The teacher uses positive verbal language through comments or statements. Specific comments such as “your work shows that you’ve done a lot of good research and preparation” or “you’ve made a positive contribution to the whole class with the completion of your project” instill in students the feeling that their work is accepted and appreciated. Nonverbal cues include such things as facial expressions, gesturing, or being in close proximity to students. Touching can also be a positive nonverbal cue when used appropriately. Be aware that some expressions may not be interpreted the same for various cultures and backgrounds. For example, touching students may not always be positive in some cultures (e.g., Indo Chinese cultures). 7. Demonstrates respect The teacher shows consideration and demonstrates respect for each student by recognizing the diversity in students and their need for individual recognition and appreciation. 43 8. Lack of ridicule/sarcasm/personal criticism The teacher does not ridicule students. The teacher avoids rudeness and sarcastic remarks such as, “When are you going to stop acting like a baby?” 9. Accommodates variety of activities The teacher should arrange the room to accommodate both large and small group activities and provide for unobstructed physical movement within the area. 10. Maintains attractive environment The teacher maintains an attractive room that promotes learning. She/he arranges attractive bulletin boards, learning centers, displays, and color to enhance the classroom environment. Indicator C2. Maximizes Time Available for Instruction 1. Begins promptly The teacher wastes no time in beginning the lesson. By beginning the lesson immediately, the teacher solicits on-task behavior from students. Students who begin promptly to engage in appropriate learning activities perform better during the lesson. 2. Minimizes loss of instructional time The teacher avoids loss of time by developing and using procedures that minimize time spent in performing nonacademic tasks. Activities and materials for the lesson have been planned and are ready for teacher and student use. 3. Establishes routines The effective teacher has established procedures to avoid wasted time by having routines for such things as checking the class roll, moving about the classroom, turning in assignments, and distributing materials. The teacher also provides smooth transitions between activities. 4. Discourages/redirects digressions During the lesson, the teacher monitors to ensure that student attention is not diverted from the activities or focus of the lesson. The teacher quickly handles situations that would cause students to focus their attention elsewhere. The teacher also discourages digressions through monitoring activities. 5. Minimizes teacher digressions The effective teacher avoids digression from the lesson. Digression occurs when a teacher discusses events or situations that are unrelated to the lesson. For example, discussion of a recent sporting event or movie unrelated to the lesson would be considered a digression. 6. Minimizes student down time The effective teacher provides students with appropriate, meaningful, and sufficient activities to be completed in the allotted time. Appropriate activities 44 are also planned for students who finish their work early. Students who have difficulty are provided relevant, secondary activities to assist them. 7. Handles interruptions The teacher handles interruptions quickly and efficiently to avoid any waste of class time. Often interruptions cannot be avoided and the teacher provides activities for students until he/she can handle the situation and get back to the lesson. Indicator C3. Manages Student Behavior to Provide Productive Learning Opportunities for All 1. Emphasizes and monitors rules/procedures The effective teacher ensures that students are aware of the rules and guidelines for their behavior. Often, students have input into making the rules and guidelines. Students need to be reminded of behavioral expectations throughout the year. A teacher comment such as, “Thank you for remembering to put your chairs back in place quietly” can be effective. 2. Keeps students on task The effective teacher avoids behavioral problems with students by keeping them on task. Students are made aware of the objectives of the lesson and of the activities to be completed within a certain amount of time. Teacher monitoring helps to ensure that students are focused. 3. Anticipates and intervenes A teacher may use student names or close proximity to prevent potential student behavior problems. The teacher is aware of the potential for misbehavior and focuses on any problem areas. 4. Uses verbal/nonverbal actions The teacher chooses the technique that will cause the least disruption to the class. At times, eye contact or close proximity to a student who misbehaves may be more effective than telling the student to be quiet and complete the work, thereby disrupting the whole class. 5. Uses reasonable sanctions When disciplining students, the effective teacher uses the least disruptive technique. Moving a student to another area in the room or speaking directly to a child who misbehaves may be more effective than making an issue of the behavior for the entire class. 6. Rewards appropriate behavior The teacher shows appreciation when students behave appropriately by rewarding that behavior. The teacher may give stickers or extra time on a favored activity, such as computer time, for students who behave well. 7. Treats students fairly and equitably The teacher ensures that rules of conduct are applied to all students in a fair and reasonable manner. 45 Indicator D1. Plans and Implements Systematic Assessments 1. Monitors student learning and activities The teacher monitors students to check for understanding of the content or material presented. The teacher may also check to determine if students understand how to perform an assigned task. The teacher may observe students, circulate through the class, or move closer to a particular student to check for accuracy or understanding. 2. Provides feedback Effective teachers provide feedback to students by recognizing their contributions/participation, by affirming correct responses, by praising specific learning behaviors, by providing specific corrective information, and by making specific recommendations for improvement. Feedback can occur immediately to student responses or by reporting on tests or homework results. Feedback assumes that a student has reacted or responded in some way or has initiated a question or comment. 3. Re-teaches when necessary Once the teacher has determined misunderstanding of the lesson or concept, he/she redirects the lesson through the use of different activities or materials that focus on the same concepts or skills. Indicator E1. Speaks Clearly, Correctly, Coherently 1. Adjusts rate, volume, pitch The teacher, in response to a student request or in recognition of the need to change, speeds up or slows down his/her rate of speaking to accommodate students. He/she also makes adjustments to volume by speaking louder or softer in response to a student request or in recognition of the need to change voice level for the purpose of stressing a point. The teacher uses inflections in recognition of the need to change voice intonation for the purpose of stressing a point. 2. Speaks fluently/uses correct pronunciations The teacher avoids excessive use of repetition of words or ideas. He/she should also avoid the use of “uhs,” “ahs,” “you know” and vague words or terms. The teacher consistently speaks distinctly and avoids mumbling or slurring of words. 3. Organizes presentation The teacher gives evidence of having organized the presentation of the lesson by avoiding gaps in the presentation or periods when he/she appears to search for the next point or statement. 46 Indicator E2. Writes Clearly, Correctly, Coherently 1. Uses correct grammar/mechanics/spelling Written communication displayed in the classroom or used in response to student work exhibits the use of correct grammar. The teacher displays correct spelling in written communication. 2. Writes legibly Students can interpret the teacher’s writing without difficulty. 3. Organizes information The teacher’s written communication reflects a systematic arrangement of the content presented. The information is organized to promote student learning. 47 THIS PAGE INTENTIONALLY LEFT BLANK 48 DOCUMENT/ARTIFACT REVIEW The purpose of the document/artifact review is to examine various artifacts and information provided by the teacher. There is no specific form of packaging required for submitting the documents (binder, file folder). However, there are organizational requirements for the teacher to follow in the presentation and written explanations or rationales of the documents/artifacts he/she provides as evidence. 1. The document entries must be easily identifiable—name of teacher, date submitted. 2. Entries must be organized around the standard and indicator for which they have been submitted. 3. A completed Mississippi Teacher Document/Artifact Entry Form specific to the standard and indicator must accompany every entry. It must be handwritten. 4. Teachers who did not teach the previous school year are exempt from submitting artifacts. Teachers will not be provided feedback on the document/artifacts until the appraisal conference. Types of Evidence The document/artifact evidence is not what the teacher plans to do, but what he/she has already done. Thus, the teacher is not to create documents/artifacts, but to present existing materials or evidence that is not obtained from the other data collection instruments. Much of the evidence provided by the teacher will probably be in document (print) form. However, computer disks, videotapes, or audiotapes are acceptable. Examples of artifacts that could be used for entries include: letters/memoranda, reports, plans, materials prepared for students, parents, or others, videotapes of faculty training sessions, training agendas, analyses of test data (paper or computer disk), feedback reports, and the like. The type of document/artifact used for each entry should address the specific standard and indicator. For example, the entry for Standard D, Indicator D4: Produces Evidence of Student Academic Growth should contain a document/artifact that shows student gains in achievement or knowledge as a result of the teacher’s intervention. This growth most likely would be specific to a unit of instruction that was taught. It should be noted that, for some students, growth might be in small amounts. For certain special-needs students, measurable growth in life skills may be appropriate evidence. 49 Mississippi Teacher Document/Artifact Entry Form Each document/artifact entry must be accompanied by a completed Mississippi Teacher Document/Artifact Entry Form specific to the standard and indicator being addressed. The form serves four purposes: 1. Description – identifies the entry by title or some designation that can be used for future reference by both the evaluator and the teacher. The description is to be concise and complete. If a document or tape is being submitted as evidence, but only a portion applies, the description prepared for the indicator should specify exactly what portion of the entry is to be reviewed. 2. Explanation – briefly explains what the entry is about. In this section, the specific items to be reviewed are to be explained. If appropriate, the explanation should include how the specific document/artifact was used. 3. Relevancy – provides the evaluator with the teacher’s thoughts about the entry’s contributions to the standard and indicator addressed. Remember that no entry is self-explanatory. The entry form must clearly explain the relevance of the whole or specific parts of the entry as evidence of the teacher’s performance in the standard and indicator. In this section, the teacher should explain why a particular document/artifact demonstrates his/her performance in the standard and indicator. For example, the teacher may begin his/her explanation for the first entry by stating, “This entry demonstrates my knowledge/skill in and ability to bring about student academic growth because …” The teacher should provide details on how the document/artifact included for this entry is relevant to the standard and indicator. Further, it is important that the document/artifact be directly linked to the teacher. It should not be something developed by others without the teacher’s leadership and active involvement. 4. Application/Follow-up – describes follow-up or changes made by the teacher. In this section, the teacher should explain any follow-up actions taken. For example, for the first entry, the teacher may indicate that he/she re-taught the concepts and skills to students who scored low or who did not meet the objectives. For the second entry, the teacher might provide an agenda or something related to a workshop he/she attended. Then the teacher might describe what was done differently in the classroom or what things were applied to his/her instruction as a result of what was learned in the workshop. 50 MISSISSIPPI TEACHER DOCUMENT/ARTIFACT ENTRY FORM (THIS FORM MUST BE HANDWRITTEN AND NOT TYPED.) Teacher:__________________ School_______________ Date of Submission: __________ Check indicator you will address on this form and attach appropriate artifacts/ documents. Indicator D4: Demonstrates academic growth Indicator F1, Artifact 1: Demonstrates that the teacher regularly engages in professional development activities that can improve his/her instruction and improve curriculum available to students Indicator F1, Artifact 2: Demonstrates that the teacher uses ideas from books, professional journals, professional organizations, and technological resources to improve instruction and learning. Indicator F2: Demonstrates that the teacher takes a role in improving the school, carrying out school improvement plan, etc. 1. Description (e.g. test results, letter from parent, newspaper article, video tape) and Explanation of Content (What is the content of the entry and who were the intended audiences?): 2. Relevancy to Indicator (State how the content of the entry relates to the indicator checked above. State why the entry is valuable evidence of your performance in this indicator. Please be specific and attach additional pages, if needed.): 3. Application/Follow-up (Explain how the student data, professional development activity, or school-improvement activity benefited students, your instruction, or the school. Attach additional page if needed.): 51 Guidelines for Document Entries Composing entire notebooks, etc. is neither necessary, nor desirable. Only one entry is to be submitted for each artifact. This does not imply the use of only one piece of paper. For example, to show continuous professional development (Standard F, Indicator F1), one sheet of paper would probably not suffice. Entries must be current. Acceptable dates are defined under each individual indicator below. Although the entry form must be handwritten, the artifacts/documents may be typed or handwritten. All are to be legible. All of the artifacts must include accompanying documentation or explanation of follow-up and/or application in the classroom of knowledge and skills gained by the teacher. Guidelines for Individual Artifacts Standard D, Indicator D4: Produces Evidence of Student Academic Growth Under His/Her Instruction Artifact 1: Document/artifact that demonstrates student academic growth This entry should be from the current school year or last school year. This entry should include documents that show that the teacher’s instruction produced academic growth in his/her students. The documents should also demonstrate the teacher’s ability to measure, analyze growth, and use the results. An example would be an analysis of pre- and post-test results for a specific unit or objective. Standard F, Indicator F1: Engages in Continuous Professional Development Artifact 1: Document/artifact demonstrating that the teacher regularly engages in professional development activities that can improve his/her instruction and improve curriculum available to students. This entry may include documents/artifacts from this school year and the last two school years. Documents older than that will not be considered. Acceptable documents/artifacts include, but are not limited to: o -CEU certificates or certificates of attendance o -Proof of enrollment/grades of college courses relevant to areas of teacher’s instruction o -E-learning courses through MDE –documentation of completion or enrollment o -Sign-in sheets and agendas from district or school professional development o -Proof of attendance at conventions, conferences, etc. 52 Standard F, Indicator F1: Engages in Continuous Professional Development Artifact 2: Document/artifact demonstrating that the teacher uses ideas from books, professional journals, professional organizations, and technological resources to improve instruction and learning This entry should be from this school year or last school year. Standard F, Indicator F2: Takes an Active Role in School Improvement Artifact 1: Document/artifact demonstrating that the teacher takes a role in improving the school, carrying out school improvement plan, etc. This entry should be from this school year or last school year. Document/artifact used here might demonstrate roles such as identifying and assisting in removing barriers to student learning; initiating activities and projects in the classroom and school that are consistent with the school improvement plan; and working collaboratively with administrators, faculty, and staff to implement school improvement. Standard E, Indicator E2: Writes clearly, correctly, and coherently There is no artifact submitted for this indicator. The score for this indicator will be based on the teacher’s written communication skills as demonstrated in all the entries submitted. Steps for Evaluating the Artifacts: Two evaluators score the documents/artifacts, and they will do so independently. After both evaluators have recorded their scores and rationales on the rubrics, they meet together and come to a consensus. There is a form in the primary evaluator’s packet for the purpose of recording the consensus score and rationales. Each evaluator: 1) Reads each rubric and reviews the teacher’s entry form and documents/artifacts. 2) Decides which score best fits the teacher’s entry. There is no need to start at 1 or 4. 3) Writes a rationale that specifically indicates why that score was awarded. Remember: If material is presented without an entry form, it is not considered. If more than one entry is provided for an indicator, the second one is not considered. The teacher is allowed to hand in more than one item to address the indicator, but all the items should be addressed in the one entry form. For example, to show continuous professional development, the teacher may hand in several certificates of completion, 53 sign-in sheets, etc. along with documentation of how the information was used in her classroom to the benefit of the students. The teacher should indicate the relevancy of all included documents on the one entry form for that document. If the teacher does not do so, the documents should be ignored. If there is no entry for an indicator, the score is 1. The rationale would be “No entry was provided for this indicator.” The entry form explains the artifacts. It is not up to the evaluator to guess why the teacher provided the documents and what they mean. Refer to the “Guidelines for Document Entry” and other information provided in the prior sections. Please see the team leaders if there are questions. Do not consult with the other evaluator until the evaluation has been independently completed. 54 MISSISSIPPI TEACHER APPRAISAL SYSTEM DOCUMENT/ARTIFACT SCORING RUBRIC Standard D. An effective teacher demonstrates knowledge and skill in assessing and facilitating student learning. PERFORMANCE LEVELS INDICATOR D4. Provides Evidence of Student Academic Growth RATING 4 Exemplary 3 Meets Standard 2 Needs Improvement 1 Unsatisfactory The evidence indicates The evidence indicates The evidence indicates The evidence indicates 1 that the teacher’s instruction produced learning (accomplishment of stated objectives and/or substantive learning gains) for all or most students for whom data were presented. that the teacher has understanding of how to compile and analyze student data to identify strengths and weaknesses. how the student data have been used to guide instruction and revise curriculum. 2 that the teacher’s instruction produced learning (accomplishment of objectives and/or substantive learning gains) for the majority (more than half) of students for whom data were presented. that the teacher knows how to compile and analyze student data to identify strengths and weaknesses. how the student data have been used to guide instruction or revise curriculum. that the teacher’s instruction produced learning (accomplishment of objectives and/or substantive learning gains) for fewer than half the students for whom data were submitted. that the teacher appears to have some understanding of how to compile and analyze student data. that the teacher has attempted to use student data to guide instruction. RATIONALE FOR SCORE: MUST BE PROVIDED. FOR ADDITIONAL SPACE, USE THE BACK OF THIS PAGE. 55 that the teacher’s instruction produced learning (accomplishment of objectives and/or substantive learning gains) for few or none of the students for whom data were submitted. that teacher appears to have little or no understanding of how to compile and analyze student data.. that the teacher has made little or no effort to use data to guide instruction. 3 4 MISSISSIPPI TEACHER APPRAISAL SYSTEM DOCUMENT/ARTIFACT SCORING RUBRIC Standard E. An effective teacher communicates effectively with students, peers, and parents/guardians. PERFORMANCE LEVELS RATING INDICATOR E2. Writes clearly, correctly, and coherently 4 Exemplary 3 Meets Standard 2 Needs Improvement 1 Unsatisfactory The documentation exhibits The documentation exhibits The documentation exhibits The documentation exhibits excellent written communication skills on the part of the teacher, i.e., construction, grammar, spelling, punctuation, capitalization, clarity, vocabulary, etc. no errors. extraordinary fluency and articulation. information organized and presented logically. adequate, but not outstanding written communication skills on the part of the teacher, i.e., construction, grammar, spelling, punctuation, capitalization, clarity, vocabulary, etc. no patterns of errors. adequate fluency and articulation. information organized and presented logically. some inadequacies on the part of the teacher’s written communication skills, i.e., construction, grammar, spelling, punctuation, capitalization, clarity, vocabulary, etc. a few patterns of errors (1-2). some problems with fluency. and articulation problems concerning the organization and logical presentation of the information. RATIONALE FOR SCORE: MUST BE PROVIDED. FOR ADDITIONAL SPACE, USE THE BACK OF THIS PAGE. 56 multiple inadequacies on the part of the teacher’s written communication skills, i.e., construction, grammar, spelling, punctuation, capitalization, clarity, vocabulary, etc. multiple patterns of errors (e.g., construction, grammar, capitalization, word usage). multiple problems in fluency and articulation. multiple problems. concerning the organization and logical presentation of the information. 1 2 3 4 MISSISSIPPI TEACHER APPRAISAL SYSTEM DOCUMENT/ARTIFACT SCORING RUBRIC Standard F. An effective teacher is committed to continual professional growth and school improvement. PERFORMANCE LEVELS Indicator F1. Engages in Continuous Professional Development . 4 Exemplary 3 Meets Standard 2 Needs Improvement 1 Unsatisfactory The evidence indicates that continuous, targeted professional development is a goal of the teacher. He/she has participated in professional development activities beyond those required by the school, school system or state and targeted at least one weak area of instruction or curriculum for improvement. employed several relevant activities and strategies from one or more sources (courses, workshops, on-line search, journal review, books, peer observations) for learning new knowledge and skills in the area(s) targeted. demonstrated application of knowledge and skills in more than one area in the classroom. provided multiple benefits to students, instruction, or the school as a result of participation in professional development activities. The evidence indicates that professional development is a priority for the teacher. He/she has participated in professional development activities beyond those required by the school, school system or state. identified useful idea(s) relevant to curriculum and instruction from at least one source. demonstrated application of knowledge and skills in the classroom. provided several benefits to students, instruction, or the school as a result of participation in professional development activities. The evidence indicates that professional development is not a high priority of the teacher. He/she has participated only in professional development activities required by the school, school system, or state. identified idea(s) with minimal relevancy to curriculum and instruction from at least one source. demonstrated only limited application of knowledge and skills gained, provided limited benefits to students, instruction, or the school as a result of participation in professional development activities. The evidence indicates that the teacher’s professional development is either not evident or is minimal. He/she has participated in a minimal number of required professional development activities. identified no ideas relevant to curriculum or instruction. demonstrated little or no application of knowledge and skills gained. provided little or no benefits to students, instruction, or the school as a result of participation in professional development activities. RATIONALE FOR SCORE: MUST BE PROVIDED. FOR ADDITIONAL SPACE, USE THE BACK OF THIS PAGE. 57 RATING 1 2 3 4 MISSISSIPPI TEACHER APPRAISAL SYSTEM DOCUMENT/ARTIFACT SCORING RUBRIC Standard F. An effective teacher is committed to continual professional growth and school improvement. PERFORMANCE LEVELS INDICATOR F2. Takes an Active Role in School Improvement RATING 4 Exemplary 3 Meets Standard 2 Needs Improvement 1 Unsatisfactory The evidence shows that the teacher has taken an active, leadership role in carrying out school improvement plans through collaboration with others (e.g., colleagues in the building, parents, the community). initiation of numerous activities in the classroom and/or school to facilitate improvement. The evidence shows that the teacher has taken an active role in carrying out school improvement plans through collaboration with others (e.g., colleagues in the building, parents, the community). initiation of some activities in the classroom and/or school to facilitate improvement. The evidence shows that the teacher has done what he/she was asked to do on behalf of school improvement, but there is little evidence of collaboration with colleagues. The evidence showed minimal initiation of activities in the classroom and/or school to facilitate improvement. The evidence does not show that the teacher has taken an active, collaborative, and initiating role in school improvement. that the teacher initiated activities in the classroom and/or school to facilitate improvement. RATIONALE FOR SCORE: MUST BE PROVIDED. FOR ADDITIONAL SPACE, USE THE BACK OF THIS PAGE. 58 1 2 3 4 MISSISSIPPI TEACHER APPRAISAL REPORT Introduction The Mississippi Teacher Appraisal Report is the instrument that summarizes performance across all standards and indicator areas of the appraisal. The report serves as the cumulative evaluation record by providing the final results of the appraisal process. The appraisal report is composed of five documents: 1) Mississippi Teacher Appraisal Report – cover sheet This cover sheet provides the teacher an explanation of the other report documents the teacher will receive. 2) Part A: Scores and Rationales On this form, the indicator scores from the interview, document/artifact review, and observations are developed into a single score for each standard with a rationale given for each score. 3) Part B: Strengths, Weaknesses, Comments and Signatures Part B provides the teacher a summary of strengths and weaknesses. Both the teacher and the evaluator may make comments about the appraisal or the conference on this form. The evaluator and the teacher sign the form to indicate that the conference was held. 4) Performance Level The standard scores are reported and averaged on this form to give a final score. The cut-score for teachers is 3.0. Pass/fail is also indicated. 5) Teacher Performance Standards Checklist All indicators and definition items deemed to be in need of improvement are checked on this form. Any teacher failing to make the cut-score of 3.0 is to use this form, along with Part A: Scores and Rationales, to write his/her individual improvement plan. 59 Completing the Mississippi Teacher Appraisal Report By Hand Part A: Scores and Rationales 1) Complete the demographic information on the form. 2) Transfer scores from the identified instruments to the appropriate open cells on the report. The left side of the form lists all of the standards and indicators and the right side lists the various instruments/data sources used in the appraisal. The darkened cells indicate that no scores are available from the instruments listed. For example, a review of the report reveals that Indicator A2, displays an open cell under only two of the columns. Therefore, the only scores that should be available for transfer are the observation scores and a score from the interview. The observation score is an average of the observation scores. Scores from observation 1, 2, and 3 (if conducted) are to be entered in the boxes to the left of the box in the observation column. For example, the score for observation 1 is to be entered in the small box labeled Ob1. The score for observation 2 is to be entered in the small box labeled Ob2. If there is a 3rd observation, the score is to be entered under Ob3. If the observations were scored a 2 and a 3, the average (2.5) would be entered in the box in the OB (observation) column. If three observations were scored 2, 2, and 3, the average would be 2.3. This is the only place in the appraisal process that averaging is used. 3) Use pattern scoring to develop a score for the standard. The evaluator reviews the indicator scores and rationales on identified instruments to develop a score for the standard. The evaluator asks two questions when reviewing the indicator scores: 1. What is the pattern of performance in this standard as suggested by the indicator scores? 2. Are there exceedingly strong or weak indicators? If so, what do the rationales for those scores suggest about their importance to the overall scoring of the standard? Note: Responding to these questions requires a review of the rationales in the various instruments. Once a score for the standard has been determined, the evaluator enters it in the space marked "Standard ‘X’ Score" at the bottom of the indicator listings. 60 4) Provide a rationale for each standard area score. The rationale is specific, obtained from the instruments in the appraisal process, and clearly indicates to the teacher the reasons or justification for the assigned scores on the report. The back of the form is to be used if more room is needed. A rationale is to be written for each standard even if all indicators were scored a 3. The rationale should give clear evidence or justification for the indicator scores. Important questions the evaluator asks when completing the rationale on the report are: 1. If the appraisal record is reviewed by an objective person, will a clear, consistent, and fair picture of the teacher’s performance emerge? 2. Will the reasons for assigned scores be consistent among standard and indicator scores, the various instruments used in the process, and the documents provided? 5) The final handwritten copy of this form should be signed by the evaluator and will be put into the MDE files. The teacher will receive a copy of Form A in an envelope on the last day of the site visit. Part B: Strengths, Weaknesses, Comments and Signatures 1) Complete the demographic information. 2) Provide a summary of the teacher’s strengths and weaknesses in the space provided. It is to be consistent with Part A and the Teacher Performance Standards Checklist. 3) The evaluator and teacher may use the comment section to make comments concerning the appraisal or the conference. 4) Both the evaluator and the teacher sign and date the form, using blue ink, to acknowledge that the conference was held. 5) This form is taken into the appraisal conference. A copy may be left with the teacher. The original with the signatures in blue is to be put into the MDE files. Performance Level 1) Complete the demographic information. 2) Transfer the standard scores from Part A to this form. The scores are added and averaged by dividing the total by the number of standards evaluated. The score is rounded to the nearest tenth. 3) Write Yes or No* in the box to indicate pass or fail. The cut-score for the teacher is 3.0. 4) A copy of this form will be put in an envelope to be given to the teacher at the end of the site visit. The original will be put into the MDE files. 61 Teacher Performance Standards Checklist 1) Complete this form for all teachers who scored a 1 or 2 on any indicator. 2) Complete the demographic information. 3) Put a checkmark next to any indicator scored a 1 or 2. Even if the standard score was a 3, any indicator scored a 1 or 2 within that standard must be checked. For example, if Standard A was scored a 3, based on an interview score of 2 for Indicator A1, and an interview score of 3 and observation score of 3 for A2, then A1 would be checked on the checklist, but Indicator A2 would not. 4) Under each indicator checked, definition items determined in need of improvement by the evaluation instruments, are also to be checked. Applicable appraisal instruments are to be used as a reference. 5) Keep in mind that the teacher will use the Teacher Performance Standards Checklist, along with Part A: Scores and Rationales, to write his/her improvement plan. Completing the Mississippi Teacher Appraisal Report with Computer Assistance Some evaluation teams use the computer to assist in completion of the appraisal documents. Many of the steps are the same. The differences are outlined as follows: As soon as the evaluator completes each instrument (observation, interview, document consensus form), he/she hands it in to the team leaders. They enter the scores on to the appraisal Part A template. The team leaders will enter identifying documentation (name, school, etc.) on Part A. This information is automatically transferred to Part B , the Performance Level form and the Teacher Performance Standards Checklist The computer automatically enters the averaged observation score. Once all scores have been entered, the team leader provides the evaluator a copy of Part A: Scores and Rationales with all recorded scores. The evaluator follows the steps for determining standard scores and writing rationales. Any indicator scoring a 1 or 2 is written in red in the box Score Rationale. These indicators are to be addressed on the Teacher Performance Standards Checklist. The evaluator then returns the form to the team leader so the standard scores and rationales can be entered. Since the template will allow for longer rationales through the size of the font, the evaluator is not limited to the size of the rationale box on the Part A: Scores and Rationales form. The final printed copy of this form should be signed by the evaluator and will be put into the MDE files along with the handwritten copy. The evaluator should also complete the Strengths and Weaknesses section of Part B and turn it into the team leader. Standard Scores and Pass/Fail will automatically be transferred to the Performance Level form. Also, indicators scored a 1 or 2 will be checked in red on the Teacher Performance Standards Checklist. Indicators scored a 3 or 4 will be marked “passed.” The evaluator will need to check definition items that are in need of improvement by hand and return it to the team leader so a typed copy can be generated. 62 The Appraisal Conference An appraisal conference should be scheduled with the teacher no later than the last day or two of the site visit, or earlier at the team leader’s discretion. The conference does NOT have to occur in the last two days. Two activities occur during this conference: 1) The evaluator discusses the strengths and weaknesses listed on Part B with the teacher. The focus is on practices and behaviors that generated identified strengths and areas of need of improvement. All indicators scored a 1 or 2 should be addressed in the conference. The Teacher Performance Standards Checklist should be used as a guide for a meaningful discussion. 2) The evaluator asks for the teacher's insights and comments on the Comments section of Part B. Both the evaluator and teacher sign and date the form in blue ink. Signatures of the teacher and evaluator indicate that the conference was held, not that the teacher is in agreement with the appraisal. Notification of Scores and Pass/Fail At the very end of the site visit, the team leaders or MDE personnel provide the teacher an envelope containing the Mississippi Teacher Appraisal Report. It includes: the cover sheet, Part A: Scores, and Rationales, the Performance Level form, and the Teacher Performance Standards Checklist. 63 THIS PAGE INTENTIONALLY LEFT BLANK 64 MISSISSIPPI TEACHER APPRAISAL REPORT Part A: Scores and Rationales Part A of the Mississippi Teacher Appraisal Report is used to provide the teacher a summary of the results of the interview, observations, and document review. It provides scores for each indicator area and an overall score for each standard. In addition, rationales for the scores are provided for each standard. A four-point scale is used: 4 - Exemplary indicates that the teacher’s performance is exemplary in this position. The teacher almost always exceeds expected standards. 3 - Satisfactory indicates that the teacher’s performance usually meets or exceeds expected standards. Areas noted can be improved, but current performance is clearly acceptable. 2 - Needs Improvement indicates that the teacher’s performance is sometimes, but not always, acceptable in this position. The teacher sometimes meets expected standards. Improvement activities are necessary to meet standards on a consistent basis. 1 - Unsatisfactory indicates that the teacher’s performance is not acceptable. Performance in this position rarely meets standards. The scores for observations are averaged to provide one indicator score. Two evaluators score the documents/artifacts individually and then reach a consensus score. The overall score for each standard is not obtained through averaging, but rather from pattern scoring. The evaluator does a pattern analysis to determine 1) the pattern of performance in this standard as suggested by the indicator scores, and 2) if there are exceedingly strong or weak indicators. Part B and the Appraisal Conference An appraisal conference is scheduled with one of the evaluators and the teacher. At this conference, the evaluator focuses the discussion on practices and behaviors that generated identified strengths and areas in need of improvement. Part B provides a summary of these strengths and weaknesses. The evaluator and teacher may write comments concerning the appraisal or the conference in the section provided. Both are to sign and date the form in blue ink to indicate the conference was held. Performance Level On the last day of the site visit, the teacher is given an envelope containing the Mississippi Teacher Appraisal Report. Scores for all indicators and standards are recorded on Part A: Scores and Rationales. The standard scores are averaged and rounded to the nearest tenth on the Performance Level form. A cut-score of 3.0 is required for a teacher to pass. Indicators and definition items that were scored as areas in need of improvement are checked on the Teacher Performance Standards Checklist. These areas are to be addressed in an individual improvement plan if the teacher scores below 3.0. Any teacher required to write an improvement plan will be given instruction and assistance by school administrators and MDE personnel. 65 Page 1 of 4 MISSISSIPPI TEACHER APPRAISAL REPORT FORM Part A: Scores and Rationales Teacher: Date: Evaluator: District: School: OB = Classroom Observation IN = Interview Standard A and Indicators DOC = Document/Artifact Review A. Plans Instruction Consistent with State Standards to Facilitate High Levels of Learning for All Students A1. Plans units of instruction that are aligned with state standards and curriculum frameworks Ob1 Ob2 A.2 Plans instruction to address individual differences in learners Score Rationale: DOC Ob3 OB = Classroom Observation IN = Interview DOC = Document/Artifact Review Delivers Instruction that Facilitates High Levels of Learning for All Students Ob1 Ob2 Ob3 Ob1 Ob2 Ob3 Ob1 Ob2 Ob3 B1. Demonstrates deep knowledge of content B2. Actively engages students in the learning process B3. Uses principles of effective teaching Score Rationale: IN STANDARD A SCORE (1-4): Standard B and Indicators B. OB STANDARD B SCORE (1-4): 66 OB IN DOC Teacher: Page 2 of 4 OB = Classroom Observation IN = Interview Standard C and Indicators C. DOC = Document/Artifact Review Organizes and Manages the Learning Environment and Maintains Student Behavior that is Conducive to Learning OB Ob1 Ob2 Ob3 Ob1 Ob2 Ob3 Ob1 Ob2 Ob3 IN DOC C1. Maintains an environment and climate conducive to learning C2. Maximizes time available for instruction C3. Manages student behavior to provide productive learning opportunities for all Score Rationale: STANDARD C SCORE (1-4): OB = Classroom Observation IN = Interview DOC = Document/Artifact Review Standard D and Indicators D. Demonstrates Knowledge and Skill in Assessing/Facilitating Student Learning Ob1 Ob2 OB Ob3 D1. Plans and implements systematic assessments D2. Considers state content standards and assessments when planning classroom assessments D3. Uses classroom assessment results D4. Produces evidence of student academic growth under his/her instruction Score Rationale: STANDARD D SCORE (1-4): 67 IN DOC Teacher: Page 3 of 4 OB = Classroom Observation IN = Interview DOC = Document/Artifact Review Standard E and Indicators E. Communicates Effectively With Students, Peers, and Parents/Guardians OB Ob1 Ob2 Ob3 Ob1 Ob2 Ob3 IN DOC E1. Speaks clearly, correctly, and coherently E2. Writes clearly, correctly, and coherently E3. Establishes effective communication processes Score Rationale: STANDARD E SCORE (1-4): OB = Classroom Observation IN = Interview DOC = Document/Artifact Review Standard F and Indicators F. Committed to Continual Professional Growth and School Improvement F1. Engages in continuous professional development F2. Takes an active role in school improvement Score Rationale: STANDARD F SCORE (1-4): 68 OB IN DOC Page 4 of 4 MISSISSIPPI TEACHER APPRAISAL REPORT FORM Part A: Scores and Rationales Certification and Attestation: I, ________________________________, a member of the School At-Risk independent evaluation team assigned to the review of the _________________________ School District, do hereby certify and attest that the above instrument is a true and correct copy of the evaluation of ________________________ completed during the review conducted ___________________________. Date(s) of Evaluation _______________________________ Evaluator’s Signature ____________________ Date 69 MISSISSIPPI TEACHER APPRAISAL REPORT FORM Part B: Comments and Signatures Teacher: Date: Evaluator: School: Appraisal Areas of Strength and Evaluator Comments: Appraisal Areas in Need of Improvement and Evaluator Comments: Teacher Comments: Evaluator Signature:_________________________________________ Date: Teacher Signature:__________________________________________ Date: Teacher's signature indicates this form has been reviewed and a conference has been held. It does not necessarily indicate agreement with the appraisal. 70 MISSISSIPPI TEACHER APPRAISAL SYSTEM PERFORMANCE LEVEL Teacher: Date: School: District: Score Standard A An effective teacher plans instruction consistent with standards to facilitate high levels of learning for all students. B An effective teacher delivers instruction that facilitates high levels of learning for all students. C An effective teacher organizes and manages the learning environment and maintains student behavior that is conducive to learning. D An effective teacher demonstrates knowledge and skill in assessing and facilitating student learning. E An effective teacher communicates effectively with students, peers, and parents/guardians. F An effective teacher is committed to continual professional growth and school improvement. Score Total: Average Score: This teacher has met the performance level cut-score: *Attached is a list of standards, indicators, and definition items that were scored as areas in need of improvement. Teachers who failed to meet the cut-score must address all checked items in an Individual Improvement Plan. Instruction and assistance by school administrators and MDE personnel will be provided 71 MISSISSIPPI TEACHER PERFORMANCE STANDARDS CHECKLIST Teacher________________________ School_______________________Date_________________ Items checked on this list are those deemed “In Need of Improvement” and are to be addressed on the teacher’s individual improvement plan. Further information and instruction will be provided by MDE personnel and/or school personnel. Standard A. An effective teacher plans instruction consistent with state standards to facilitate high levels of learning for all students. Indicator A1. Plans units of instruction that are aligned with state standards and curriculum framework develops/selects long-range goals and measurable, short-term objectives consistent with state standards and curriculum frameworks selects content that will lead to accomplishment of goals and objectives selects instructional resources and activities that will enable students to accomplish objectives maintains records of what has been taught and uses these data in further planning Indicator A2. Plans instruction to address individual differences in learners identifies individual differences among students within a class (achievement, ability, interests, motivation, learning styles) addresses differences by varying content addresses differences by varying materials addresses differences by varying questions addresses varying learning styles of students Standard B. An effective teacher delivers instruction that facilitates high levels of learning for all students Indicator B1. Demonstrates deep knowledge of content presents accurate and up-to-date information presents developmentally appropriate content relates content to life experiences presents content in logical pattern/sequence relates content to other subject areas uses relevant examples identifies misconceptions/lack of understanding presents lesson/content in alternative ways responds knowledgeably/accurately to questions Indicator B2. Actively engages students in the learning process communicates with students at their levels of language and comprehension encourages participation ensures equitable participation 72 elicits responses encourages student sharing uses student ideas/responses seeks elaboration refers to other students engages students in knowledge/hypotheses varies roles in instructional process develops/selects activities that require student engagement stimulates and encourages higher-order thinking at appropriate developmental levels Indicator B3. Uses Principles of Effective Teaching makes sure that students know what they are doing and why they are doing it relates lesson to objectives/emphasizes main idea sequences lesson establishes relationships adjusts lesson, when appropriate provides guided practice provides independent practice provides review/summarization uses variety of resources/materials and technology uses multiple representations and explanations promotes higher-order thinking Standard C. An effective teacher organizes and manages the learning environment and maintains student behavior that is conducive to learning. Indicator C1. Maintains an environment and climate conducive to learning organizes available space, materials, and equipment to facilitate learning creates and maintains a colorful, inviting classroom or other environment accommodates variety of activities displays acceptance, appreciation, and encouragement of all students acknowledges and celebrates student successes establishes time lines for completion of tasks/assignments establishes standards for academic performance holds students accountable/encourages quality indicates confidence uses verbal/nonverbal enthusiasm uses positive verbal language/nonverbal cues demonstrates respect does not use ridicule/sarcasm/personal criticism Indicator C2. Maximizes time available for instruction begins promptly manages time allotted to activities and adjusts time when necessary 73 minimizes loss of instructional time establishes routines and transitions and manages them efficiently discourages/redirects digressions minimizes teacher digressions minimizes student down time handles interruptions Indicator C3. Manages student behavior to provide productive learning opportunities for all establishes, emphasizes, and monitors rules/procedures monitors the classroom and makes necessary adjustments to prevent undesirable behavior addresses behavior problems when they occur keeps students on task anticipates and intervenes uses verbal/nonverbal actions uses reasonable sanctions rewards appropriate behavior treats students fairly and equitably Standard D. An effective teacher demonstrates knowledge and skill in assessing and facilitating student learning. Indicator D1. Plans and implements systematic assessments develops an assessment plan when planning an instructional unit uses a variety of assessment methods, including appropriate pre- and post-assessments and informal collection of data during instruction aligns assessments with unit objectives, content, and instruction analyzes results of assessments and uses these results when planning instruction monitors student learning and activities provides feedback re-teaches when necessary Indicator D2. Considers state content standards and assessments when planning classroom assessments aligns classroom assessments with state content standards and curriculum frameworks aligns classroom assessments with content and format of state assessments in appropriate ways Indicator D3. Uses classroom assessment results uses results to determine student progress and learning uses results to improve curriculum and instruction 74 Indicator D4. Produces evidence of student academic growth under his/her instruction provides assessment data and/or student work that show accomplishment of desired objectives by most students as a result of instruction provides assessment data and/or student work that show substantial learning gains by most, if not all, students as a result of instruction Standard E. An effective teacher communicates effectively with students, peers, and parents/guardians. Indicator E1. Speaks clearly, correctly, and coherently uses standard speech pronounces words correctly uses vocabulary and style appropriate to level of audience speaks fluently without hesitations adjusts rate, volume, pitch when necessary organizes presentation Indicator E2. Writes clearly, correctly, and coherently spells words correctly uses correct grammar and mechanics writes legibly uses vocabulary and style appropriate to level of audience organizes written information Indicator E3. Establishes effective communication processes develops and maintains systematic, two-way communication with students, school personnel, and parents/guardians uses a variety of methods to keeps parents/guardians informed of curriculum focus and instructional activities and progress of students provides student assessment results to those who can benefit from them (e.g., students, parents, other faculty) seeks to involve parents/guardians in their children’s learning and in the school demonstrates respect and consideration for all those with whom he/she interacts Standard F. An effective teacher is committed to continual professional growth and school improvement. Indicator F1. Engages in continuous professional development. establishes a systematic program to improve weaker areas of performance takes formal coursework to enhance knowledge and skills, if appropriate participates in school, system, and state-sponsored professional development programs uses ideas from books, professional journals, professional organizations, and technological resources to improve instruction and learning applies knowledge gained from these experiences to the classroom to the benefit of students 75 Indicator F2. Takes an active role in school improvement identifies and assists in removing barriers to student learning initiates activities and projects in the classroom and school that are consistent with the school improvement plan works collaboratively with administrators, faculty, and staff to implement the school improvement plan OTHER: 76 NOTES __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ 77