1 The Impact of Feedback Given to Primary School Teachers on their Teacher Practices Rocío Grissel Estrada Castro TUM School of Education, Technical University of Munich ED2302: Planning and Implementation of Research Works in Classroom Research I M.A. Annika Diery and Dr. Despoina Georgiou July, 2020 2 Abstract Research problem: Teacher evaluation has been introduced around the world with the intent to improve teaching. However, in the literature on teacher evaluation, often findings reveal critical accounts about the effectiveness of feedback in teacher evaluation for teacher practices and student achievement. Method: TALIS 2018 dataset was used, were primary school teachers (ISCED level 1) N=7,767 from Australia, France and Japan were selected for the study. The study tested two hypotheses (1) Feedback primary school teachers receive have a positive impact on their teacher practices; (2) There are significant differences in the extent of feedback primary school teachers receive and their teacher practices between Australia, France and Japan. Two scales were created: TDSCFB (α = .70), to measure the extent of teachers received feedback and TT3PRA (α = .70) to assess Teachers’ Best Practices, based on Hattie (2009). Results: There is a medium positive correlation r (6098) = .24, p< .001. between the extent of feedback primary school teachers receive and their teacher practices; ANOVA raised that the Teachers’ Practices F(2, 6095) = 299.27, η2= .06 and the extent of Feedback Teachers Received F(2, 7764) = 244.09, η2= .09, had significant differences (p<.001) in the means between the countries, with exception of France-Japan comparison of means in Tukey’s post-hoc test (-.01, .6, p>.05) in the Teachers’ Practices scale. Conclusions: The extent of feedback teachers receive is related to teacher practices, feedback, whether provided by internal or external bodies is recommended to be provided as part of teachers’ evaluation models. Key words: Feedback, teacher practices, teacher evaluation. 3 The Impact of Feedback given to Primary School Teachers on their Teacher Practices As the world changes, expectations placed on education also change. Education institutions are responsible for the future and development of millions of children worldwide. Providing every child with good education opportunities means providing every child with a good teacher (UNICEF, 2018), yet teachers do not develop alone but organizations, internal and external bodies (Standaert, 2000) in general play a relevant role in their professional development. Specifically, teacher evaluation aims to improve performance as part of a continuous improvement cycle in the professional growth and to achieve this, the use of feedback is essential (Tuytens & Devos, 2016). The OECD comparative data and indicator on teacher policies Teachers Matters (OECD, 2005) posit that teachers need feedback on their performance to help them identify how to better shape and improve their teaching practice. There is a wealth of evaluation instruments and literature on the importance of external (Campbell & Levin, 2009; Cizek, 2000; Earl & Fullan, 2003; Heritage & Yeagley, 2005; Vanhoof & Petegem, 2007; Visscher & Coe, 2003), and internal (Darr, 2018) evaluations and the assumed positive influence on teaching and learning to be gained from feeding back the results to teachers. However, few empirical studies (Hellrung & Hartig, 2013; Tresch, 2007) have studied whether such feedback provided to teachers at the school level actually benefits the teacher practices. Feedback One of the main components of various teacher evaluation models such as the Marzano Focused Teacher Evaluation Model (Marzano & Toth, 2013), The Danielson Model (Danielson, 2007), CIPP Model (Stufflebeam, 2003) and other researched based models (Chang & Wang, 2016) is feedback given to teachers based on their assessment in order to make a change in their teacher practices (Chang & Wang, 2016; Tuytens & Devos, 2016). Feedback is also relevant for the Teacher Evaluation Reform (Donaldson, 2016) and it is 4 expected to have an impact on teacher practices and student achievement. Carless (2015) mentioned that feedback is used it to enhance the quality of the work of teachers or learning strategies. Feedback should aim towards improvement in students’ learning or refocus either the teacher’s or the learner’s actions to achieve a goal (Dawson, Henderson, Mahoney, Phillips, Ryan, Boud & Molloy, 2018). Teacher educators must encourage teachers to use effective teaching practices and one technique for increasing use of effective practices is providing feedback to teachers (Scheeler, Ruhl & McAfee, 2004). According to the Teaching and Learning International Survey TALIS, 2018 (OECD, 2019a) there are different forms to provide feedback to teachers such as: Observation of classroom teaching. Observation frameworks usually focus on what is understood by a good teacher, a description of his pedagogical exercise, observation criteria and their corresponding indicators (Muller, Volante, Grau & Preiss, 2014). Student survey responses. Student ratings are the most influential measure of teaching effectiveness, active participation and meaningful input from students (Chen & Hoshower, 2010) but there is no evidence that the use of survey was making any contribution to improving the overall quality of teaching (Kember, Leung & Kwan, 2010). Assessment of content knowledge. Teachers’ knowledge about the subject matter to be learned or taught (Koehler & Mishra, 2009). External results of students. Hellrung & Hartig (2012) found than less than 20 percent of teachers take or plan to take measures as a result of the feedback. Time intervals between testing and feedback were found to be too long for teachers to make effective use of it. School based and classroom-based results. Includes performance results or test scores. Black & William (2009) define the classroom as formative, where evidence about student achievement is used by teachers to make decisions about the next steps in instruction. 5 Self-assessment. According to Rueda Beltrán & Diá z Barriga (2004) portfolio assessment can be useful about how is teaching conducted, in order to plan the pertinent changes. Video-feedback intervention involved teachers watching video-recordings of their own teaching and self-evaluating their teaching practices (Blood Pinter, East & Thrush, 2015). Entities who give feedback Based on Scheeler, Ruhl & McAfee (2004) and the TALIS Questionnaire (OECD, 2019a) we got: (1) External individual or bodies, (2) school principal or members of the school (school management team) and (3) other colleagues within the school (not school management team). The leadership of the Director and management team is particularly relevant (Tuytens & Devos, 2016). Verhaeghe, Vanhoof, Valcke & Van Petegem, (2009) found that teachers expected feedback within the same school year as when tests had been taken, in order to support low-scoring pupils and Donaldson (2016) mentioned teachers valued the feedback they received in Teacher Evaluation from more frequent observations and post-observation conferences but they noted that principals were struggling to find the time to provide this feedback. Findings indicate (Day, Sammons, Leithwood, & Hopkins, 2011; Verhaeghe, et al, 2009) that teachers can get stuck in the process of feedback use by authorities, thus not achieving the goal of improving teaching quality (OECD, 2005). Teaching Practices The educational researcher John Hattie wrote a book based on the meta study Visible Learning (Hattie, 2009). He linked student outcomes to several highly effective teacher practices from which Self-Reported Grades, Classroom Discussion, Teacher Clarity and Feedback belonged to the top 5 of the highest effect sizes. Self-Reported Grades. Students self-report their own grades meaning they are more likely to be successful than other learners as they will be the active element in their learning. 6 Once a student has performed at a level that is beyond their own expectations, he or she gains confidence in his or her learning ability. Classroom Discussion. The teacher stops lecturing and students get together to discuss an important issue. This allows students to learn from each other. Teacher clarity. Teacher clarifies the purpose and learning goals at the beginning of a new unit and provides explicit criteria on how students can be successful. Feedback. According to Hattie and Timperley (2007) feedback is one of the most powerful influences on learning and achievement. Feedback on task, process and self-regulation level is far more effective than on the Self-level. Japan, Australia and France With regards to the countries included in the study, it is known that Japan, Australia and France have been in the 30-top list for at least 5 years in their ranking on several global educational studies such as PISA (OECD, 2005). For Japan, the main indicator contributing to its success has been the development of students between age 5 to 14. France is having the highest enrollment rate of early childhood and The Education Index listed Australia as the second-highest in the world (Human development data, 2017). Research Questions Which are the characteristics of the types and entities that provide feedback to primary school teachers during the last 12 months in Australia, France and Japan? To what extent does the feedback primary school teachers receive during the last 12 months has an impact on their teaching practices in Australia, France and Japan? H1: Feedback primary school teachers received from internal and external bodies during the last 12 months have a positive impact on their teacher practices. H0: Feedback primary school teachers received from internal and external bodies during the last 12 months have no impact on teacher practices. 7 Are they significant differences of the extent of feedback received by primary school teachers and teacher practices between Australia, France and Japan? H2: There are significant differences in the extent of feedback received by primary school teachers and teacher practices between Australia, France and Japan. H0: There are no significant differences in the extent of feedback received by primary school teachers and teacher practices between Australia, France and Japan. Methods Sample The data was obtained from the Teaching and Learning International Survey, TALIS 2018 dataset (OECD,2019b). Primary school teachers (ISCED level 1) N=7,767 from Japan n=3308 (Age M= 50-59), Australia n=3030 (Age M= 30-39) and France n=1429 (Age M= 40-49) were selected for the study. 73.1% Females and 23.4% Males with a formal education completed of ISCED 2011 from Levels 3 to 8, M=6. From the survey population of each country the method of sample selection was systematic proportional to number of teachers, with overlap control used for the ISCED level 2 and the TALIS-PISA link field trial sample. Design The data was collected with a cross-sectional survey. According to the TALIS Report 2018 (OECD, 2019b) main survey data collection took place between September and December 2017 for Southern Hemisphere participants and March to May 2018 for Northern Hemisphere participants (with some participants starting early in February and some extending into July 2018). During the main survey, 91.3% of the respondents completed the survey online and 8.7% completed it on paper (OECD, 2019b). Instrument The Teaching and Learning International Survey (TALIS) 2018 Teacher Questionnaire (OECD, 2019b) was used for the study. The instrument consisted of fifty-eight items (α .70). For the purposes of this research, we used the variable IDCNTRY for the 8 country categories and created two scales to measure our dependent variables. The first, with eighteen items (TDSCFB, α = .70) to obtain the extent of Teachers Received Feedback in the last twelve months, item example: “I have received observation of classroom teaching by external individuals or bodies”. In order to create the TDSCFB Scale, a sum of the original TT3G29 scale positive written items was made and recoding was needed. The interpretation of the scale would be ordinal, from 0= No feedback received to 18= Feedback received by the three entities in the 6 forms of feedback1 (OECD, 2019a). The second scale, with ten items (TT3PRA, α = .70) was created to assess the Teachers’ Best Practices, based on Hattie (2009). The subscales consisted of Teacher Clarity (TEACLA, α = .67) item example: “I set goals at the beginning of instruction”; Student Feedback (STUDFB, α = .44) e.g. “I observe students when working on particular tasks and provide immediate feedback”; Classroom Discussion (CLADIS, α = .54) e.g. “I have students work in small groups to come up with a joint solution to a problem”; Self-Reported Grades (SRGRAD, α = .42) e.g. “I let students evaluate their own progress”. The scale has a 4-point Likert Scale from 1= “Never or almost never” to 4= “Always”. Analyses The statistical software used for the analyses was GNU PSPP 2017 (MacOS Test Version). With regards to the first research question, frequency tables with central tendencies were obtained for the Teachers Received Feedback to detail the characteristics of the types and entities that provide feedback to primary school teachers in Australia, France and Japan. Frequencies of type of feedback teachers receive from internal and external bodies was provided. Descriptives were obtained for the Teachers’ Practices. For the test of our first hypothesis, we tested if the sample was parametric or non-parametric via Shapiro-Wilk normality test and based on the results, Pearson Correlation was run to see if Teachers 1 Detailed in Feedback section, p.1. See appendix figure 1 for feedback item detail. 9 Received Feedback and Teachers’ Practices were related. Finally, for the test of our second hypothesis, the means of Teachers Received Feedback and Teachers’ Practices between Australia, France and Japan were compared with a one-way ANOVA test and Tukey B posthoc tests were applied in order to spot the differences of means. Additionally, to obtain a more detailed study, another ANOVA and Tukey B was made to detect differences in the subscales between the countries Teachers’ Practices. Results For our variables, Teachers Received Feedback and Teachers’ Practices, ShapiroWilk normality test was applied, obtaining a non-parametric sample p>.05. However, due to the size of the sample and the robustness of the parametric tests, we decided to run parametric tests for the test of our hypothesis. For our first research question, the descriptive analyses identified that 3% of teachers in Australia, 6% in Japan and 14% in France reported that they had never received feedback in their schools, compared with the average 10% (OECD, 2019a). In Australia, 81.5% of teachers reported that the feedback they received has improved their teaching practice; In Japan, 87.2% and in France, a smaller proportion of teachers with 60,7%. Table 1 Percentage of Forms of Feedback Most Commonly Received by Teachers* Form of Feedback Country Australia France Japan Observation of classroom teaching 38.4 29.9 48.7 Student survey responses 15.2 13.5 20.6 Assessment of content knowledge 17.5 14.3 25.5 External results of students 27.9 19.6 27.2 School-based and classroom-based results 31.8 15.7 26.5 Self-assessment of work 20.5 5.8 20.9 Note. The table represents how observation of classroom teaching is the most common form of feedback and student survey responses one of the less common. *In the last twelve months. 10 Table 2 Percentage of Chosen Entities Teachers Have Received Feedback From* Entity Country Australia France Japan External individuals or bodies 7.6 31.75 11.5 School principal or school management team 37.8 8.8 40.8 Other colleagues within school or not school 29.45 8.95 32.4 management team Note. The table highlight that France receive more feedback from external individuals or bodies in comparison with Australia and Japan who receive more feedback from the school principal or school management team. *In the last twelve months. With regards to the Teachers’ Practices (TT3PRA) characteristics, we obtained Teacher Clarity (M=3.14), Feedback Students Receive (M=2.36), Self-Reported Grades (M=2.24) and Discussion (M=2.50), were Teacher Clarity is a frequent teacher practice and the last three are occasionally realized teacher practices in Australia, France and Japan2. Concerning the relationship between the extent of Feedback Teachers Received (TDSCFB) and Teachers’ Practices (TT3PRA), we confirmed that there is a positive correlation r (6098) = .24, p < .001. between Feedback and Teachers’ Practices and rejected the null hypothesis3. However, the correlation obtained is a small correlation according to Cohen (1992) but medium, according to Gignac and Szodorai (2016), who recommends correlations .20 to .29 to be medium based on his meta-analytically derived correlations research. Based on the context of the research, we decided to consider r=.24 as a medium correlation as it is an unusual effect on social sciences. For the test of our second hypothesis, ANOVA raised that the Teachers’ Practices F(2, 6095) = 299.27, η2= .06 (medium effect size) and the extent of Feedback Teachers Received F(2, 7764) = 244.09, η2= .09 (high effect size), had significant differences (p<.001) in the means between Australia, France and Japan. In order to discover the precise mean 2 3 For detailed Teachers’ Practices means by country, see Figure 2. See appendix figure 2 11 differences between the groups, we applied Tukey’s test and found out that in both variables, the countries have significant differences (p<.001), with exception of France-Japan comparison of means (-.01, .6, p>.05) in the Teachers’ Practices scale. Figure 1 Mean differences in the Feedback Teachers Receive by countries. Extent of feedback received* Feed b ack Teach ers Recei ve 6 4 2 0 Feedback Australia France Mean 2,98 4,53 Japan 5,1 Note. The extent is from 0 to 18. The second ANOVA results detected the differences in the subscales between the countries Teachers’ Practices, revealing that Teacher Clarity F(2, 6150) = 47.43, η2= .015 (small effect size); Feedback Students Receive F(2, 6173) = 612.9, η2= .017 (small effect size); Self-Reported Grades F(2, 6190) = 317.74, η2= .1 (medium effect size); and Discussion F(2, 6189) = 34.07, η2= .06 (medium effect size) had significant differences in all of the Tukey B post-hoc test comparisons (p<.05). Figure 2 Mean differences in the Teachers’ Practices subscales by countries. Frequency of Teacher Practices* Teacher Practices subscales 4 3,5 3 2,5 2 1,5 1 Australia France Japan TC 3,3 3 3,14 FB 3,1 2,8 2,5 SRG 2,5 1,8 2,2 D 2,7 2,5 2,6 Note. TC=Teacher Clarity, FB=Feedback, SRG= Self-reported grades, D=Discussion *The frequency is based on a 4-point Likert scale. 12 Discussion The study contributes to close the gap on the lack of research in the assumed positive influence that feedback (Campbell & Levin, 2009; Earl & Fullan, 2003; Heritage & Yeagley, 2005; Vanhoof & Petegem, 2007; Visscher & Coe, 2003; Darr, 2018), often cited as a main component on teacher evaluation models (Marzano & Toth, 2013; Danielson, 2007; Chang & Wang, 2016) actually have on teacher practices. Furthermore, answer the question whether feedback is effective at all. We support Hattie & Timperley (2007) study, who mentioned feedback is one of the most powerful influences on learning and achievement and Tuytens & Devos (2016), where teacher evaluation aims to improve performance as part of a continuous improvement cycle in the professional growth and affirm that to achieve it, the use of feedback is relevant, if not essential. This study also supports Scheeler, Ruhl & McAfee (2004) who mentioned that one technique for increasing use of effective practices is providing feedback to teachers. As Donaldson (2016) mentioned, feedback is also relevant for the Teacher Evaluation Reform and it was expected to have an impact on teacher practices and it is used to enhance the quality of the work of teachers (Carless, 2015). In summary, France has a lower extent of feedback teachers receive and frequency of teacher practices, which can be reflected in studies such as PISA (OECD, 2005) were Australia and Japan have better overall scores. With regards to the specific teacher practices being related to feedback, we could not obtain a correlation due to a limitation in the data input but obtained significant differences of means by countries. Hattie (2009) included more teacher practices than the TALIS Questionnaire included, but would be of interest to develop a Questionnaire that include all of them, as it is a robust meta-analysis study. Even if we cannot assure that one specific type of feedback has more impact than others, we observed in the results how observation of classroom teaching is the most common form of feedback, Donaldson (2016) mentioned 13 teachers valued the feedback they received in Teacher Evaluation from more frequent observations and post-observation conferences, and observed that student survey responses was one of the less common -possibly related to the lack of evidence that the use of survey was making any contribution to improving the overall quality of teaching (Kember, Leung & Kwan, 2010)- showing significant differences in the feedback teachers receive by countries, were Japan receive the more extent of feedback, followed by Australia and France, which has a smaller proportion of teachers who reported that feedback they received has improved their teaching practice (OECD, 2019a). This could be due to authorities who get stuck in the process of feedback use (Day, Sammons, Leithwood, & Hopkins, 2011; Verhaeghe, et al, 2009), thus not achieving the goal of improving teaching quality (OECD, 2005). The entities who give feedback are also related to teacher practices but specific correlations could not be obtained. Based on the descriptives, France received more feedback from external individuals or bodies in comparison with Australia and Japan who receive more feedback from the school principal or school management team, Tuytens & Devos (2016) sustained that the leadership of the Director and management team is particularly relevant in providing feedback. Overall feedback teachers received is related to teacher practices, but further research is needed to detail specifically which of the teacher practices have gained a more positive impact. Based on the results of this study, we found that Teacher Clarity, Feedback, Selfreported grades and Discussion, which are on the highest top effect size list related to student achievement (Hattie, 2009) have significant differences of means by countries and France have a lower frequency of best teacher practices compared with Australia and Japan. Highlighting Self-reported grades in France, who have a significant lower frequency compared to Australia and Japan. It is relevant that if France is having the highest enrollment rate of early childhood (Human development data, 2017) they also have a high use of teacher 14 practices in primary school education and precisely Self-reported grades, as it is the teacher practice who has the higher effect size in student achievement (Hattie, 2009). This study is significant to teachers, internal and external evaluators, school management team and educational practitioners. We conclude that results provided an understanding of how feedback is related to teacher practices in an overall set and between countries. Further research is needed to affirm the correlation between specific forms of feedback/entities and the impact on an isolated teacher practice. Donaldson (2016) and other researchers (Dawson, Henderson, Mahoney, Phillips, Ryan, Boud & Molloy, 2018) mentioned that feedback is expected to be relevant in student achievement, being this a good future research topic, as it is not as explored as an isolated component of the teacher evaluation models. We recommend further research to include more groups of study with different backgrounds, since a limitation of this study is that selected countries are on the toplist of educational systems. Diversification of backgrounds could lead to clearer results in the differences between feedback received and teacher practices. 15 Appendix Figure 1 Feedback TALIS 2018 Questionnaire items detail Figure 2 Feedback Teachers Received Scatterplot of correlation between Feedback Teachers Received and Teachers’ Practices Teachers’ Practices Note. Representation of the positive correlation r (6098) = .24, p < .001. 16 References Blood Pinter, E., East, A. & Thrush, N. (2015). Effects of a Video-feedback Intervention on Teachers’ Use of Praise. Education and Treatment of Children. West Virginia University Press, 38(4), 451-472. 10.1353/etc.2015.0028 Boud, D. & E. Molloy. (2013). Rethinking Models of Feedback for Learning: The Challenge of Design. Assessment & Evaluation in Higher Education. 38 (6), 698–712. https://doi.org/10.1080/02602938.2012.691462 Campbell, C. & Levin, B. (2009). Using data to support educational improvement. Educational Assessment, Evaluation, and Accountability, 21(1), 47–65. http://dx.doi.org/10.1007/s11092-008-9063-x. Carless, D. (2015). Excellence in University Assessment: Learning from Award-Winning Practice. London: Routledge. Chang, T., & Wang, H. (2016). A Multi Criteria Group Decision-making Model for Teacher Evaluation in Higher Education Based on Cloud Model and Decision Tree. Eurasia Journal of Mathematics, Science and Technology Education, 12(5), 1243-1262. https://doi.org/10.12973/eurasia.2016.1510a Chen, Y. & Hoshower, L. (2003). Student evaluation of teaching effectiveness: an assessment of student perception and motivation. Assessment and Evaluation in Higher Education, 28(1), 71-88. DOI: 10.1080/02602930301683 Chen, Y. & Hoshower, L. (2010). Student Evaluation of Teaching Effectiveness: An assessment of student perception and motivation. Assessment & Evaluation in Higher Education, 28(1), 71-88. https://doi.org/10.1080/02602930301683 Cizek, G. J. (2000). Pockets of resistance in the assessment revolution. Educational Measurement: Issues and Practice, 19(2), 16–23. http://dx.doi.org/10.1111/j.17453992.2000.tb00026.x. Cohen, J. (1992). Statistical power analysis. Current Directions in Psychological Science, 1(3), 98–101. https://doi.org/10.1111/1467-8721.ep10768783 Cohen, J. & Goldhaber, D. (2016). Building a More Complete Understanding of Teacher Evaluation Using Classroom Observations. Educational Researcher. 45(6), 378-387. https://doi.org/10.3102/0013189X16659442 Danielson, C. (2007). Enhancing professional practice: A framework for teaching (2nd ed.). Alexandria, VA: Association for Supervision and Curriculum Development. Darr, C. (2018). A return to assessment for learning: back to the future. Research information for teachers online first. New Zealand council for educational research. https://doi.org/10.18296/set.0102 Dawson, P., Henderson, M., Mahoney, P., Phillips, M., Ryan, T., Boud, D. & Molloy, E. (2018). What makes for effective feedback: staff and student perspectives. Assessment & Evaluation in Higher Education, 44(1), 25-36. https://doi.org/10.1080/02602938.2018.1467877 Day, C., Sammons, P., Leithwood, K., & Hopkins, D. (2011). Successful school leadership: linking with learning and achievement. Maidenhead: Open University Press. ISBN 978 1 84478 822 4. Donaldson, M. (2016). Teacher Evaluation Reform: Focus, Feedback, and Fear. Educational Leadership, 8(73),72-76. ISSN-0013-1784 Douglas, J. & Douglas, A. (2006). Evaluating Teaching Quality. Quality in Higher Education. 12(1), 3-13. https://doi.org/10.1080/13538320600685024 Earl, L. & Fullan, M. (2003). Using data in leadership for learning. Cambridge Journal of Education, 33(3), 383–394. http://dx.doi.org/10.1080/0305764032000122023. 17 Fischer, C., Fishman, B., Dede, C., Eisenkraft, A., Frumin, K., Foster, B., Lawrenz, F., Jurist Levy, A. & McCoy, A. (2018). Investigating relationships between school context, teacher professional development, teaching practices, and student achievement in response to a nationwide science reform. Teaching and Teacher Education, 72, 107-121. https://doi.org/10.1016/j.tate.2018.02.011 Gignac, G. E., & Szodorai, E. T. (2016). Effect size guidelines for individual differences researchers. Personality and Individual Differences, 102, 74–78. https://doi.org/10.1016/j.paid.2016.06.069 Hattie, J. (2009). Visible Learning: A Synthesis of Over 800 Meta-analyses Relating to Achievement. Routledge: New York. Hattie, J. & H. Timperley (2007). The Power of Feedback. Review of Educational Research, 77 (1), https://doi.org/10.3102/003465430298487 Hellrung,K. & Hartig, J. (2013). Understanding and using feedback - A review of empirical studies concerning feedback from external evaluations to teachers. Educational Research Review, 9, 174-190. http://dx.doi.org/10.1016/j.edurev.2012.09.001 Heritage, M. & Yeagley, R. (2005). Data use and school improvement: Challenges and prospects. Yearbook of the National Society for the Study of Education, 104(2), 320– 339. http://dx.doi.org/10.1111/j.1744-7984.2005.00035.x. Hill, H. & Grossman, P. (2013). Learning from Teacher Observations: Challenges and Opportunities Posed by New Teacher Evaluation Systems. Harvard Educational. 83(2), 371-384. https://doi.org/10.17763/haer.83.2.d11511403715u376 Hounsell, D. (2003) The evaluation of teaching in H. Fry, S. Ketteridge, and S. Marshall. A handbook for teaching and learning in higher education: enhancing academic practice. London: KoganPage. Human Development Data (2017). Human Development Reports, United National Development Program, 14 September 2018, retrieved 6 August 2019. Kember, D., Leung, D. & Kwan, K. (2010). Does the use of student feedback questionnaires improve the overall quality of teaching?. Assessment and Evaluation in Higher Education, 27(5), 411-425. https://doi.org/10.1080/0260293022000009294 Koehler, M. J. & Mishra, P. (2009). What is technological pedagogical content knowledge? Contemporary Issues in Technology and Teacher Education, 9(1), 60-70. https://doi.org/10.1177/002205741319300303 Marzano, R. J., Toth, M. D. (2013). Teacher Evaluation That Makes a Difference: A New Model for Teacher Growth and Student Achievement. ASCD. ISBN 978-1-4-41661573-6. Mihaly, K. & Mccaffrey, D. (2015). Grade-Level Variation in Observational Measures of teacher Effectiveness. Designing Teacher Evaluation Systems, pp.7-49. https://doi.org/10.1002/9781119210856.ch2 Muller, M., Volante, P., Grau, V., & Preiss, D. (2014). Desarrollo de habilidades de observación en la formación de liderazgo escolar a través de videos de clases. Psykhe (Santiago), 23(2), 1–12. OECD. (2019a), Education at a Glance 2019: OECD Indicators, OECD Publishing, Paris, https://doi.org/10.1787/f8d7880d-en OECD. (2018), Effective Teacher Policies: Insights from PISA, PISA, OECD Publishing. http://dx.doi.org/10.1787/9789264301603-en OECD. (2019b), TALIS 2018 Technical Report. OECD Publishing, Paris, https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&cad=rja&ua ct=8&ved=2ahUKEwj_nZrd1aXqAhWt16YKHc3RDSwQFjAAegQIBxAB&url=http s%3A%2F%2Fwww.oecd.org%2Feducation%2Ftalis%2FTALIS_2018_Technical_R eport.pdf&usg=AOvVaw0qE3L8YuUaJT-aKhGYdow- 18 Rueda Beltrán, M. & Diá z Barriga, F. (2004). El portafolios docente como recurso innovador en la evaluación de los profesores. La evaluación de la docencia de en la universidad, México: UNAM. Scheeler, M. C., Ruhl, K. L. & McAfee, J. K. (2004). Providing Performance Feedback to Teachers: A Review. Teacher Education and Special Education: The Journal of the Teacher Education Division of the Council for Exceptional Children. 27(4), 396-407. https://doi.org/10.1177/088840640402700407 Shute, V. J. (2008). Focus on Formative Feedback. Review of Educational Research, 78 (1). https://doi.org/10.3102/0034654307313795 Standaert, R. (2000). Inspectorates of Education in Europe: A Critical Analysis. Flanders: Ministry of Education. Stufflebeam D.L. (2003) The CIPP Model for Evaluation. In: Kellaghan T., Stufflebeam D.L. (eds) International Handbook of Educational Evaluation. Kluwer International Handbooks of Education, vol 9. Springer, Dordrecht. https://doi.org/10.1007/978-94010-0309-4_4 Tuytens, M. & Devos, G. (2016). The role of feedback from the school leader during teacher evaluation for teacher and school improvement. Teachers and Teaching: theory and practice, 23(1), 6-24. https://doi.org/10.1080/13540602.2016.1203770 UNICEF. (2018), Education Global Annual Results Report 2018, UNICEF Publishing, New York, https://www.unicef.org/media/55331/file Vanhoof, J. & Petegem, P. (2007). Matching internal and external evaluation in an era of accountability and school development: Lessons from a flemish perspective. Studies in Educational Evaluation, 33(2), 101–119. http://dx.doi.org/10.1016/j.stueduc.2007.04.001. Verhaeghe, G., Vanhoof, J., Valcke, M. & Van Petegem, P. (2009). Using school performance feedback: perceptions of primary school principals, School Effectiveness and School Improvement. http://dx.doi.org/10.1080/09243450903396005 Visscher, A. J. & Coe, R. (2003). School performance feedback systems: Conceptualisation, analysis, and reflection. School Effectiveness and School Improvement, 14(3), 321– 349. http://dx.doi.org/10.1076/sesi.14.3.321.15842.