NASDAP Conference 2013
Prof. Eileen Piggot-Irvine
Eileen.Piggotirvine@royalroads.ca

SESSION OUTLINE
• My own rationale for appraisal!
• What’s happening in varied countries?
• Barriers to effectiveness
• A model for effective appraisal that ensures a ‘productive approach’
• Distilling the most important principles
• Collaborative re-design of own school appraisal systems with a view to incorporating the prioritised principles

THE CASE FOR RECOGNISING EFFECTIVE
TEACHING
From Looney (2012 – a superb paper):
“Students from low-income families may benefit the most from learning with very effective teachers (Nye et al., 2004; Sanders &
Rivers, 1996) and those with a succession of high-quality teachers may potentially achieve the same levels as their peers from highincome families (Motoko et al., 2007). On the other hand, Sanders and Rivers (1996) found that the residual effects of learning with poor quality teachers were long lasting, and that students assigned to more effective teachers in later grades were unable to compensate for earlier gaps .” (p.440)

BEING HONEST ABOUT MY STAND
We need great teachers and we need to know that the best teaching is happening in our schools.
In my opinion, a leader who does not want to know whether great teaching is happening, or not, is potentially robbing our children of learning.

USA
The US uses the term ‘evaluation’ as the description for overall appraisal activity. Approaches vary for individual states.

FRANCINE EDWARDS, AMERICAN FEDERATION OF
TEACHERS, 2012 KEYNOTE ADDRESS TO AERA ….
In US overall, worrying trends, standardised testing, firing and moving teachers in poorly performing schools instead of looking at the context. In a low performing Ohio school, all teachers were requested to re-sit their teacher entrance exam.
40% of teacher evaluation evidence is based on test score evidence.
Inadequate evaluation systems miss a prime opportunity to improve practice .
Evaluation (appraisal) systems need more than student achievement scores
– need information on how to further improve (feedback).
An example effective system is in Toledo - based on continuous improvement, has professional standards, measures student achievement
(but not only achievement).

FROM USA RESEARCHERS
“… a technically valid and reliable assessment framework will not by itself ensure the improvement of teaching … In our experience, the assessment of teacher quality fails more often because of organizational neglect than because of technical deficiencies. In particular, school districts have typically not done a good job of managing the relational and political aspects of the process.
”
(Donaldson & Donaldson, 2012)

THE MOVE TOWARDS PERFORMANCE
PAY IN THE US
Widescale incentives (Teacher Incentive Fund, TIF) to implement performance pay.
Extensively evaluated
(Humphreys et al., 2012), but:
“… existing knowledge about how to make performance pay a transformative reform is relatively sparse.
” (p.61)
General advice and long checklists of components that must be in place was not enough to overcome the uneven implementation from one school to the next, even within the same TIF grant holder.

MOVEMENT TOWARDS USE OF VALUE-ADDED
MEASURES (VAM)
VAM are designed to evaluate student test score gains from one year to the next.
A cautionary tale from Darling-Hammond et al. (2012):
“Research indicates that value-added measures of student achievement tied to individual teachers should not be used for high-stakes, individual-level decisions, or comparisons across highly dissimilar schools or student populations.
” (p. 14)
Extensive research has shown:
1. Value-added models of teacher effectiveness are inconsistent.
2. Teachers’ value-added performance is affected by the students assigned to them.
3. Valueadded ratings can’t disentangle the many influences on student progress.

AND MORE ON VAM
Lissitz (2012), an excellent summary paper qualified with strong statistical analysis.
Lissitz is skeptical, pessimistic and advising caution about
VAM.

BUT THERE IS HOPE AND PROGRESS BEYOND VAM!
National Board for Professional Teaching Standards (NBPTS)
– created standards that are grounded in student learning.
Interstate New Teacher Assessment and Support Consortium
(INTASC) translated NBPTS for beginning teachers - adopted by over 40 states for initial teacher licensing and have become basis for evaluation that is much more stable than VAM.
The hope is that these will extend to the evaluation process for all teachers .

HOPEFUL SYSTEMS CONT.
The Measures of Effective Teaching (MET) Project - developed a number of tools (observations, video of teachers) supplemented with other artifacts of practice (lesson plans, assignments, etc.) – evidence is scored against standards of effective teaching.
Teacher Advancement Program (TAP), based on NBPTS and INTASC standards as well as the Conneticut standards-based assessment rubrics: evaluate teachers 4-6 times a year using master/mentor teachers or principals certified in a rigorous four-day training. Teachers look at and evaluate videotaped teaching episodes using the rubric, then dialogue with mentor and plan ongoing growth.
Schools provide professional development, mentoring, and classroom support to help teachers meet these standards.

So… a mix of good and not so good exists in the US. It is highly variable, but there is hope in these latest trends.

UK
In the UK ‘performance management’ is the term applied to appraisal

In Britain a national system exists and there are direct links to salary (Bubb & Hoare, 2001).
Forrester (2011) - movement from appraisal (backward looking) to goal setting focused performance management (forward looking) with a performance related pay (PRP) component. The intent of the movement to
PRP was to incentivise teachers. Forrester suggests rather that it has been a millstone.

Recent calls in the UK for strengthening the emphasis on student learning outcomes eg Moreland (2011):
What we should be doing, in my view, is to base judgements on the responsibility the individual teacher takes for her/his own professional development and the learning and progress made within lessons … In this way, we may begin to move away from the concept of performance management as a once a year, paper-chasing exercise and move towards a process of managing performance aimed at improving the classroom experience for both teachers and students alike.
(p.24)

CANADA - BC

, The Victoria Times Colonist August 28, 2012
“If you want to press a hot button inside or outside the hallowed halls of public education, teacher evaluation would be the one almost guaranteed to light up the "Emergency!" display on every teachers' union office desk .”

A TOPIC FILLED WITH POLITICIAN AND
UNION CLASH
British Columbia Liberal leadership candidate
Kevin Falcon proposed the idea of merit pay for educators. The B.C.
Teachers’ Federation – predictably - was not in favour.
The current system gives most responsibility for teacher evaluation to principals, who are guided by language in collective agreements.
But that's not working, George Abbott, Education Minister (2011) said:
" The assessment process, I hear from most parties, is not a satisfactory one. Most assessment is done by principals and vice-principals and I believe that they feel there's much room for improvement ."

IN BC - A TWIN PROCESS
Teacher evaluation is a formal cyclical review (3 yearly,
Christian schools 4 yearly) conducted by a principal or vice principal. The collective agreement includes well-defined procedures for this retrospective model. Evaluation generally is not seen as an opportunity for reflection and growth, but as a potential impingement on teachers’ professional competence.
Teacher assessment is a more collaborative process that can include self-assessment, goalsetting and planning as well as external feedback.

TEACHER EVALUATION IS ALMOST TOTALLY
SUMMATIVE
The guidelines for evaluation include:
1. Evaluations shall take place:
- At the request of the teacher (T)
- When initiated by the Administrator (A)
- At a mutually agreeable time (M)
2. The teacher shall have the opportunity to select the observation times.
3. At least X days prior to evaluation, the teacher and the evaluator will meet and discuss evaluation criteria, timelines, etc.
4. Reports shall focus mainly on the teacher’s area of expertise
5. The teacher will receive a draft copy of the report and have the opportunity to propose changes prior to filing the final report
6. The teacher will have an opportunity to submit written commentary of the final report, which will be filed along with all copies of the final report a report written by the administrator that includes a rating of either satisfactory or unsatisfactory.

TEACHER ASSESSMENT IS ALMOST TOTALLY
FORMATIVE
A minority of school districts in BC use a professional growth plan model as part of their assessment process.
However, this process and use of a growth plan model, is not supported by the BCTF (the union).
Because growth plans are developed in consultation with a principal and because they may link to school or district goals, they are seen to impinge on professional autonomy.
You can’t win!!

APPRAISAL EFFECTIVENESS GENERALLY
Donaldson & Donaldson (2012, p. 77-), suggest the five steps to effective systems include:
Step 1: Include teachers in designing the performance evaluation system.
Step 2: Protect opportunities to learn and grow.
Step 3: Hone principals’ and coaches’ skills at observing and consulting with teachers.
Step 4: Build time for teacher evaluation into principals ’ workloads.
Step 5: Make instructional improvement a district priority.
See how this was done in NZ in Piggot-Irvine (2010).

NZ

An activity “… intended to benefit both the individual and organisation by leading to affirmation that performance expectations are being met, and to the identification of areas for improvement”
(Piggot-Irvine & Cardno, 2005, p. 12)

Appraisal fits within a larger performance management cycle. Performance management (a macro-descriptor) encompasses all of the micro-processes associated with personnel.
Performance management covers the functions that begin when a staff member enters the organisation through to their exit.
(Piggot-Irvine, 2003)

Exit
Interview
Vacancy
Recruitment selection
Performance
Management
Induction
Mentoring
Resignation
Development
Appraisal
Discipline and
Competence

Barriers to effectiveness overall in appraisal abound. Perillo
(2006) sums up many of these when stating:
“… employee conflict and competition, the undermining of teamwork, disproportionate employee responsibility for poor performance and perceptions of bureaucratic control … poor recall of development goals, confusion and doubts about the purpose and value of the process, criticism about the time lost by participating in it …” (p.365)

Just one example of barriers:
Mather and Siefert (2011) think a strong focus on accountability
(tighter control) has created an environment where:
“… senior managers have employed … close supervision of task content and its execution. As a result the professionals have resisted collectively and formally through unions, informally in the common rooms and individually through grievance, absenteeism, increased instrumentalism and dull compliance in the job .” (p.26)
Mather and Siefert continue - such activity displaces professional self-regulation, undermines high trust relations and educational autonomy.

PERFORMANCE PAY IN NZ
“To hold schools and teachers accountable for differences in attainment of social groups is particularly unfair and unreasonable” (Ivan Snook, in Hislop, 2012, p.1)
“… but the general policy approach continues to emphasise that it is underperforming teachers and schools that are at fault” (Hislop, 2012, p.1)
Hislop believes that the Minister’s agenda with improving the appraisal system is to link the accountability component to pay.

BUT THERE IS GREAT HOPE!
Although Forrester (2011) reminds us that:
“… little is known of actual costs let alone the extent to which performance management has contributed to ‘improvement’, ‘efficiency’ and ‘excellence’...” (p.8)
… it is my experience that appraisal can achieve multiple advances in clarity, focus and improved performance and relationships as positive outcomes if a developmental and accountability balance is maintained.

• Key elements of an effective systems
• Strengthening student learning outcomes via ‘deep’ goal setting.
• Establishing and maintaining trust.

Elements of Appraisal Effectiveness (Piggot-Irvine, 2003)
Beyond the Superficial – On-
Going and In-depth
Based on
Mutual Respect
Clear Guidelines
Based on Objective,
Informative, Data
Transparent and
Confidential
Criteria
For
Effective
Appraisal
Independent from
Disciplinary Aspects
An Educative Process
Trust Generating
Integrated Development and
Accountability
Well Resourced with
Training and Time

PLUS ….. making links to enhanced student learning! Sinnema and Robinson (2007) - appraisal role in promoting teacher inquiry focused on teaching (an input) that strengthens the student learning focus (an output). Has been a weak focus on student learning in appraisal due to:
Low priority placed on student learning or achievement nationally in statements of teaching standards;
Limited examination in appraisal of how teaching is impacting on students;
A compliance orientation predominating in appraisal with a lack of specific/challenging goals being set; and
The belief that appraisal is widely seen as an opportunity for celebration and support may prevent confronting of assumptions and requests for evidence in the process.

The ‘deep’ goal setting that has strongly been promoted previously by Piggot-Irvine (2003) and subsequently
Piggot-Irvine and Cardno (2005), supports the strengthening of the student learning outcomes focus that
Sinemma and Robinson have subsequently emphasised.
The goal setting session addresses this issue.

DEEP GOAL SETTING SHOULD CASCADE FROM
THE STRATEGIC OBJECTIVES OF THE SCHOOL
Cascading alignment between school wide vision and goals, team and individual goals

In onethird of schools, the principal’s appraisal was linked to targets or strategic goals.
In 21 percent of schools, both the principals’ and the teachers’ appraisals were linked to targets or achievement goals.
Many schools did not recognise that appraisal is a key part of a systematic approach to improving the effectiveness of teaching and outcomes for students. Improvements are more likely to occur when key elements are connected and support each other. Appraisal provides an opportunity for leaders and teachers to inquire together into the impact of teaching on student learning.

Many of the schools that linked school and appraisal goals demonstrated consistently high performance in the six dimensions of a successful school. These included a commitment to all students achieving, and a sense of shared responsibility for student learning, as well as a systematic approach including:
• using data to identify student and staff needs
• setting school goals
• linking staff goals to the school goals
• planning and accessing related professional learning
• establishing processes where teachers discuss strategies to accelerate progress
• monitoring the impact on student learning to identify effective practices.

My Thinking!
The Appraisal Inquiry Model (AIM)
An ongoing , inclusive (strongly so with the ‘productive’ intent), situationally sensitive, teacher and student learning oriented Model.

A FURTHER AND PERHAPS MOST IMPORTANT KEY
FEATURE OF EFFECTIVENESS
If a national appraisal system is to proceed … it must be about teacher development leading to more effective inclass performance … must have a basis of peer review in a climate of trust …properly resourced … Brian
McEntee (Listener, 9 June, 2012: p.9)

Non-defensive (non avoiding, non-controlling) openness and honesty are the underpinning values needed throughout appraisal (Argyris, 2003;
Piggot-Irvine, 2012). These LEAD TO trust development.
- Openness and trust do not result from an emphasis on needing to be nice or liked (this creates avoidance ).
- Non-defensive strategies are easy to employ in non-contentious situations but this is not the case when there are difficult messages to be delivered, as is sometimes the case in appraisal.
- In such tricky situations we often unknowingly resort to defensive behaviours including the avoidance of confronting issues.
42

TWO DOMINANT DEFENSIVE STRATEGIES
AVOIDANCE
-
MIXING MESSAGES
- SOFT-SELLING
CONTROL
- PREDETERMINING OUTCOMES
- LEAPING THE LADDER OF INFERENCE
- EASING-IN - MAKING DECISIONS WITHOUT CHECKING
- NOT REVEALING THINKING - OFFERING SOLUTIONS
- ENGAGING IN SLL - OVER-ADVOCATING OWN POSITION

NON-DEFENSIVE = PRODUCTIVE STANCE
DLL
 Surfacing underlying issues and dealing with them
Reflection in Action (rather than just on Action)
 Reflecting and responding during rather than after action
Staying low on the Ladder of Inference
 Withholding judgments until after dialogue
Engaging in ‘Dialogue’ based conversations
44

A PRODUCTIVE STANCE – DIALOGUE STEPS
Advocacy
 Stating the issues/recommendations, backed by evidence
Inquiry
 Inviting challenge of position, views, evidence
 Checking what views others hold
 Receiving others’ views non-defensively
 Summarizing mutual understanding of issues
Gaining Internal Commitment
 Seeking bi or multi-lateral (genuine two or multi-sided) solutions
Monitoring
 Taking joint responsibility for planning, implementation and monitoring
45

DISTILLING RECURRING PRINCIPLES
Group Task: in would
From what that you have heard that is happening in varied countries, collectively brainstorm what you suggest are the 8 most important principles
(eg high ownership by appraisees) that need to be present for an appraisal system to be effective.
Next, prioritise the principles in terms of how much they would influence appraisal effectiveness, with the first in the list being most influential.
Record the prioritised list on the large sheets of paper provided.

A LITTLE MORE ABOUT THE TOP PRIORITIES

RE-DESIGN OF OWN SCHOOL APPRAISAL SYSTEMS
Now that you have a reasonably clear idea about what is important for effectiveness, its time to think about how to apply/incorporate these principles into your own school appraisal system.
Complete the following activities in your group:
1.
Create a model that would fit your own school.
2.
Outline the values that underpin the model.
3.
Describe the key elements (e.g. 2 sentences per element)
You will be presenting your model to the whole group.

REFERENCES
Bubb, S. & Hoare, P. (2001).
Performance management: Monitoring teachers in the primary school.
London: David Fulton.
Darling-Hammond, L., Amrein-Beardsley, A., Haertel, E., & Rothstein, J. (2012). Evaluating teacher evaluation.
Phi Delta Kappan, 93 (6), 8-15.
Donaldson, M..L., & Donaldson, G.A. (2012). Strengthening Teacher Evaluation: What District Leaders Can Do. To cultivate high-quality
teaching, school leaders need to focus on the human side of teacher assessment. Educational Leadership, May, 78-82.
Education Review Office (2013). Board Employment Responsibilities: Linking Charter Targets to Appraisal in Primary Schools (February
2013), http://www.ero.govt.nz/National-Reports/Board-Employment-Responsibilities-Linking-Charter-Targets-to-Appraisal-in-Primary-
Schools-February-2013/Overview
Forrester, G. (2011). Performance management in education: milestone or millstone? Management in Education 25 (1), 5 –9. DOI:
10.1177/0892020610383902
Hislop, D. (2012). Is performance pay just over the horizon?
Education Today, 2 , 1.
Humphrey, D.C., Gallagher, H.A., Yee, K.M., Gross, K., Campbell, A.Z., Cassidy, L.J., & Mitchell, N.M. (2012). Teacher Incentive Fund: First
Implementation Report. U.S. Department of Education Office of Planning, Evaluation and Policy Development Policy and Program Studies
Service.
Lissitz, R.W. (2012). The Evaluation of Teacher and School Effectiveness Using Growth Models and Value Added Modeling:
Hope Versus Reality. AERA Presentation, Vancouver, May.
Looney, J. (2011 ) Developing High-Quality Teachers: Teacher evaluation for improvement European Journal of Education , 46 (4), 440-455.
Mather, K. & Siefert, R. (2011). Teacher, lecturer or labourer? Performance management issues in education.
Management in Education 25 (1),
26-31. DOI: 10.1177/0892020610383902
Moreland, J. (2011). Managing performance for effective classrooms.
Management in Education 25 (1), 21-25. DOI: 10.1177/0892020610383902
Perillo, S. (2006). Practice enhancement: Optimising teaching performance in schools.
International Journal of Educational Management, 20 (5),
365-379.
Piggot-Irvine, E. (2003). Key features of appraisal effectiveness.
International Journal of Educational Management, 17 (4), 170-178.
Piggot-Irvine, E. & Cardno, C. (2005).
Appraising performance productively: Integrating accountability and development.
Auckland: Eversleigh
Publishing.
Piggot-Irvine, E. (2010). One school’s approach to overcoming resistance and improving appraisal: Organizational learning in action.
Educational Management Administration & Leadership 3 8(2) 229 –245.
Piggot-Irvine, E. (2010). Confronting evaluation blindness: Evidence of influence of action science based feedback.
American Journal of
Evaluation , 31(3), 314-325, DOI: 10.1177/1098214010369251.
Piggot-Irvine, E. (2012). Creating authentic collaboration: A central feature of effectiveness. In O. Zuber-Skerritt, Action research for sustainable development in a turbulent world (pp.89-107).
Bingley, UK: Emerald.
Sinemma, C.E.L. & Robinson, V.M.J. (2007). The leadership of teaching and learning: Implications for teacher evaluation.
Leadership and Policy in Schools, 6 , 319-343.