Appraisal - The Conference Team

advertisement

APPRAISAL: WE ALWAYS

WERE AHEAD OF THE

PLAY AND THE WORLD

IS JUST CATCHING UP!

NASDAP Conference 2013

Prof. Eileen Piggot-Irvine

Eileen.Piggotirvine@royalroads.ca

SESSION OUTLINE

My own rationale for appraisal!

What’s happening in varied countries?

Barriers to effectiveness

A model for effective appraisal that ensures a ‘productive approach’

Distilling the most important principles

Collaborative re-design of own school appraisal systems with a view to incorporating the prioritised principles

THE CASE FOR RECOGNISING EFFECTIVE

TEACHING

From Looney (2012

– a superb paper):

“Students from low-income families may benefit the most from learning with very effective teachers (Nye et al., 2004; Sanders &

Rivers, 1996) and those with a succession of high-quality teachers may potentially achieve the same levels as their peers from highincome families (Motoko et al., 2007). On the other hand, Sanders and Rivers (1996) found that the residual effects of learning with poor quality teachers were long lasting, and that students assigned to more effective teachers in later grades were unable to compensate for earlier gaps

.”

(p.440)

BEING HONEST ABOUT MY STAND

We need great teachers and we need to know that the best teaching is happening in our schools.

In my opinion, a leader who does not want to know whether great teaching is happening, or not, is potentially robbing our children of learning.

USA

The US uses the term ‘evaluation’ as the description for overall appraisal activity. Approaches vary for individual states.

FRANCINE EDWARDS, AMERICAN FEDERATION OF

TEACHERS, 2012 KEYNOTE ADDRESS TO AERA ….

In US overall, worrying trends, standardised testing, firing and moving teachers in poorly performing schools instead of looking at the context. In a low performing Ohio school, all teachers were requested to re-sit their teacher entrance exam.

40% of teacher evaluation evidence is based on test score evidence.

Inadequate evaluation systems

miss a prime opportunity to improve practice

.

Evaluation (appraisal) systems need more than student achievement scores

– need information

on how to further improve (feedback).

An example effective system is in Toledo - based on continuous improvement, has professional standards, measures student achievement

(but not only achievement).

FROM USA RESEARCHERS

“… a technically valid and reliable assessment framework will not by itself ensure the improvement of teaching

… In our experience, the assessment of teacher quality fails more often because of organizational neglect than because of technical deficiencies. In particular, school districts have typically not done a good job of managing the

relational

and political aspects of the process.

(Donaldson & Donaldson, 2012)

THE MOVE TOWARDS PERFORMANCE

PAY IN THE US

Widescale incentives (Teacher Incentive Fund, TIF) to implement performance pay.

Extensively evaluated

(Humphreys et al., 2012), but:

“… existing knowledge about how to make performance pay a transformative reform is relatively sparse.

(p.61)

General advice and long checklists of components that must be in place was not enough to overcome the uneven implementation from one school to the next, even within the same TIF grant holder.

MOVEMENT TOWARDS USE OF VALUE-ADDED

MEASURES (VAM)

VAM are designed to evaluate student test score gains from one year to the next.

A cautionary tale from Darling-Hammond et al. (2012):

“Research indicates that value-added measures of student achievement tied to individual teachers should not be used for high-stakes, individual-level decisions, or comparisons across highly dissimilar schools or student populations.

(p. 14)

Extensive research has shown:

1. Value-added models of teacher effectiveness are inconsistent.

2. Teachers’ value-added performance is affected by the students assigned to them.

3. Valueadded ratings can’t disentangle the many influences on student progress.

AND MORE ON VAM

Lissitz (2012), an excellent summary paper qualified with strong statistical analysis.

Lissitz is skeptical, pessimistic and advising caution about

VAM.

BUT THERE IS HOPE AND PROGRESS BEYOND VAM!

National Board for Professional Teaching Standards (NBPTS)

– created standards that are grounded in student learning.

Interstate New Teacher Assessment and Support Consortium

(INTASC) translated NBPTS for beginning teachers - adopted by over 40 states for initial teacher licensing and have become basis for evaluation that is much more stable than VAM.

The hope is that these will extend to the evaluation process for all teachers

.

HOPEFUL SYSTEMS CONT.

The Measures of Effective Teaching (MET) Project - developed a number of tools (observations, video of teachers) supplemented with other artifacts of practice (lesson plans, assignments, etc.)

– evidence is scored against standards of effective teaching.

Teacher Advancement Program (TAP), based on NBPTS and INTASC standards as well as the Conneticut standards-based assessment rubrics: evaluate teachers 4-6 times a year using master/mentor teachers or principals certified in a rigorous four-day training. Teachers look at and evaluate videotaped teaching episodes using the rubric, then dialogue with mentor and plan ongoing growth.

Schools provide professional development, mentoring, and classroom support to help teachers meet these standards.

So… a mix of good and not so good exists in the US. It is highly variable, but there is hope in these latest trends.

UK

In the UK

‘performance management’ is the term applied to appraisal

In Britain a national system exists and there are direct links to salary (Bubb & Hoare, 2001).

Forrester (2011) - movement from appraisal (backward looking) to goal setting focused performance management (forward looking) with a performance related pay (PRP) component. The intent of the movement to

PRP was to incentivise teachers. Forrester suggests rather that it has been a millstone.

Recent calls in the UK for strengthening the emphasis on student learning outcomes eg Moreland (2011):

What we should be doing, in my view, is to base judgements on the

responsibility the individual teacher takes for her/his own professional development

and the learning and progress made within lessons

… In this way, we may begin to move away from the concept of performance management as a once a year, paper-chasing exercise and move towards a process of managing performance aimed at improving the classroom experience for both teachers and students alike.

(p.24)

CANADA - BC

“Teacher evaluations a hot-button topic”

Geoff Johnson

, The Victoria Times Colonist August 28, 2012

“If you want to press a hot button inside or outside the hallowed halls of public education, teacher evaluation would be the one almost guaranteed to light up the "Emergency!" display on every teachers' union office desk

.”

A TOPIC FILLED WITH POLITICIAN AND

UNION CLASH

British Columbia Liberal leadership candidate

Kevin Falcon

proposed

the idea of merit pay for educators. The B.C.

Teachers’ Federation – predictably - was not in favour.

The current system gives most responsibility for teacher evaluation principals, who are guided by language in collective agreements.

to

But that's not working, George Abbott, Education Minister (2011) said:

"

The assessment process, I hear from most parties, is not a satisfactory one. Most assessment is done by principals and vice-principals and I believe that they feel there's much room for improvement

."

IN BC - A TWIN PROCESS

Teacher evaluation

is a formal cyclical review (3 yearly,

Christian schools 4 yearly) conducted by a principal or vice principal. The collective agreement includes well-defined procedures for this retrospective model. Evaluation generally is not seen as an opportunity for reflection and growth, but as a potential impingement on teachers’ professional competence.

Teacher assessment is a more collaborative process that can include self-assessment, goalsetting and planning as well as external feedback.

TEACHER EVALUATION IS ALMOST TOTALLY

SUMMATIVE

The guidelines for evaluation include:

1. Evaluations shall take place:

- At the request of the teacher (T)

- When initiated by the Administrator (A)

- At a mutually agreeable time (M)

2. The teacher shall have the opportunity to select the observation times.

3. At least X days prior to evaluation, the teacher and the evaluator will meet and discuss evaluation criteria, timelines, etc.

4. Reports shall focus mainly on the teacher’s area of expertise

5. The teacher will receive a draft copy of the report and have the opportunity to propose changes prior to filing the final report

6. The teacher will have an opportunity to submit written commentary of the final report, which will be filed along with all copies of the final report a report written by the administrator that includes a rating of either satisfactory or unsatisfactory.

TEACHER ASSESSMENT IS ALMOST TOTALLY

FORMATIVE

A minority of school districts in BC use a professional growth plan model as part of their assessment process.

However, this process and use of a growth plan model, is not supported by the BCTF (the union).

Because growth plans are developed in consultation with a principal and because they may link to school or district goals, they are seen to impinge on professional autonomy.

You can’t win!!

APPRAISAL EFFECTIVENESS GENERALLY

Donaldson & Donaldson (2012, p. 77-), suggest the five steps to effective systems include:

Step 1: Include teachers performance evaluation system.

in designing the

Step 2: Protect opportunities to learn and grow.

Step 3: Hone principals’ and coaches’ skills at observing and consulting with teachers.

Step 4: Build time principals

’ workloads.

for teacher evaluation into

Step 5: Make instructional improvement a district priority.

See how this was done in NZ in Piggot-Irvine (2010).

NZ

An activity

“… intended to benefit both the individual and organisation by leading to affirmation that performance expectations are being met, and to the identification of areas for improvement”

(Piggot-Irvine & Cardno, 2005, p. 12)

Appraisal fits within a larger

performance management

cycle. Performance management (a macro-descriptor) encompasses all of the micro-processes associated with personnel.

Performance management covers the functions that begin when a staff member enters the organisation through to their exit.

(Piggot-Irvine, 2003)

Exit

Interview

Vacancy

Recruitment selection

Performance

Management

Induction

Mentoring

Resignation

Development

Appraisal

Discipline and

Competence

Barriers to effectiveness overall in appraisal abound. Perillo

(2006) sums up many of these when stating:

“… employee conflict and competition, the undermining of teamwork, disproportionate employee responsibility for poor performance and perceptions of bureaucratic control … poor recall of development goals, confusion and doubts about the purpose and value of the process, criticism about the time lost by participating in it

…”

(p.365)

Just one example of barriers:

Mather and Siefert (2011) think a strong focus on accountability

(tighter control) has created an environment where:

“… senior managers have employed … close supervision of task content and its execution. As a result the professionals have resisted collectively and formally through unions, informally in the common rooms and individually through grievance, absenteeism, increased instrumentalism and dull compliance in the job

.”

(p.26)

Mather and Siefert continue - such activity displaces professional self-regulation,

undermines high trust relations

and educational autonomy.

PERFORMANCE PAY IN NZ

“To hold schools and teachers accountable for differences in attainment of social groups is particularly unfair and unreasonable”

(Ivan Snook, in Hislop, 2012, p.1)

“… but the general policy approach continues to emphasise that it is underperforming teachers and schools that are at fault”

(Hislop, 2012, p.1)

Hislop believes that the Minister’s agenda with improving the appraisal system is to link the accountability component to pay.

BUT THERE IS GREAT HOPE!

Although Forrester (2011) reminds us that:

“… little is known of actual costs let alone the extent to which performance management has contributed to

‘improvement’, ‘efficiency’ and

‘excellence’...”

(p.8)

… it is my experience that appraisal

can

achieve multiple advances in clarity, focus and improved performance and relationships as positive outcomes if

a developmental and accountability balance

is maintained.

WHAT IS NEEDED? WHAT IS EFFECTIVE?

Key elements of an effective systems

Strengthening student learning outcomes via ‘deep’ goal setting.

Establishing and maintaining trust.

Elements of Appraisal Effectiveness

(Piggot-Irvine, 2003)

Beyond the Superficial – On-

Going and In-depth

Based on

Mutual Respect

Clear Guidelines

Based on Objective,

Informative, Data

Transparent and

Confidential

Criteria

For

Effective

Appraisal

Independent from

Disciplinary Aspects

An Educative Process

Trust Generating

Integrated Development and

Accountability

Well Resourced with

Training and Time

-

-

-

-

PLUS ….. making links to enhanced student learning! Sinnema and Robinson (2007) - appraisal role in promoting teacher inquiry focused on teaching (an input) that strengthens the student learning focus (an output). Has been a weak focus on student learning in appraisal due to:

Low priority placed on student learning or achievement nationally in statements of teaching standards;

Limited examination in appraisal of how teaching is impacting on students;

A compliance orientation predominating in appraisal with a lack of specific/challenging goals being set; and

The belief that appraisal is widely seen as an opportunity for celebration and support may prevent confronting of assumptions and requests for evidence in the process.

The ‘deep’ goal setting that has strongly been promoted previously by Piggot-Irvine (2003) and subsequently

Piggot-Irvine and Cardno (2005), supports the strengthening of the student learning outcomes focus that

Sinemma and Robinson have subsequently emphasised.

The goal setting session addresses this issue.

DEEP GOAL SETTING SHOULD CASCADE FROM

THE STRATEGIC OBJECTIVES OF THE SCHOOL

Cascading alignment between school wide vision and goals, team and individual goals

2013 ERO REPORT

In onethird of schools, the principal’s appraisal was linked to targets or strategic goals.

In

21 percent of schools, both the principals’ and the teachers’ appraisals were linked to targets or achievement goals.

Many schools did not recognise that appraisal is a key part of a systematic approach to improving the effectiveness of teaching and outcomes for students. Improvements are more likely to occur when key elements are connected and support each other. Appraisal provides an opportunity for leaders and teachers to inquire together into the impact of teaching on student learning.

ERO (2013)

Many of the schools that linked school and appraisal goals demonstrated consistently high performance in the six dimensions of a successful school. These included a commitment to all students achieving, and a sense of shared responsibility for student learning, as well as a systematic approach including: using data to identify student and staff needs setting school goals linking staff goals to the school goals planning and accessing related professional learning establishing processes where teachers discuss strategies to accelerate progress monitoring the impact on student learning to identify effective practices.

My Thinking!

The Appraisal Inquiry Model (AIM)

An ongoing

, inclusive (strongly so with the ‘productive’ intent), situationally sensitive, teacher and student learning oriented Model.

A FURTHER AND PERHAPS MOST IMPORTANT KEY

FEATURE OF EFFECTIVENESS

If a national appraisal system is to proceed … it must be about teacher development leading to more effective inclass performance … must have a basis of peer review in

a climate of trust

…properly resourced … Brian

McEntee (Listener, 9 June, 2012: p.9)

HOW DO WE DEVELOP TRUST?

-

Non-defensive (non avoiding, non-controlling) openness and honesty are the underpinning values needed throughout appraisal (Argyris, 2003;

Piggot-Irvine, 2012). These LEAD TO trust development.

- Openness and trust do not result from an emphasis on needing to be nice or liked (this creates

avoidance

).

- Non-defensive strategies are easy to employ in non-contentious situations but this is not the case when there are difficult messages to be delivered, as is sometimes the case in appraisal.

- In such tricky situations we often unknowingly resort to defensive behaviours including the avoidance of confronting issues.

42

TWO DOMINANT DEFENSIVE STRATEGIES

-

AVOIDANCE

MIXING MESSAGES

- SOFT-SELLING

CONTROL

- PREDETERMINING OUTCOMES

- LEAPING THE LADDER OF INFERENCE

- EASING-IN - MAKING DECISIONS WITHOUT CHECKING

- NOT REVEALING THINKING - OFFERING SOLUTIONS

- ENGAGING IN SLL - OVER-ADVOCATING OWN POSITION

NON-DEFENSIVE = PRODUCTIVE STANCE

Employing:

DLL

Surfacing underlying issues and dealing with them

Reflection

in

Action (rather than just on

Action)

Reflecting and responding during rather than after action

Staying low on the Ladder of Inference

Withholding judgments until after dialogue

Engaging in ‘Dialogue’ based conversations

44

A PRODUCTIVE STANCE – DIALOGUE STEPS

Advocacy

Stating the issues/recommendations, backed by evidence

Inquiry

Inviting challenge of position, views, evidence

Checking what views others hold

Receiving others’ views non-defensively

Summarizing mutual understanding of issues

Gaining Internal Commitment

Seeking bi or multi-lateral (genuine two or multi-sided) solutions

Monitoring

Taking joint responsibility for planning, implementation and monitoring

45

DISTILLING RECURRING PRINCIPLES

Group Task: in would

From what that you have heard that is happening in varied countries, collectively brainstorm what you suggest are the 8 most important

principles

(eg high ownership by appraisees)

that need to be present for an appraisal system to be effective.

Next, prioritise the principles in terms of how much they would influence appraisal effectiveness, with the first in the list being most influential.

Record the prioritised list on the large sheets of paper provided.

A LITTLE MORE ABOUT THE TOP PRIORITIES

RE-DESIGN OF OWN SCHOOL APPRAISAL SYSTEMS

Now that you have a reasonably clear idea about what is important for effectiveness, its time to think about how to apply/incorporate these principles into your own school appraisal system.

Complete the following activities in your group:

1.

2.

3.

Create a

model

that would fit your own school.

Outline the

values

that underpin the model.

Describe the

key elements

(e.g. 2 sentences per element)

You will be presenting your model to the whole group.

REFERENCES

Bubb, S. & Hoare, P. (2001).

Performance management: Monitoring teachers in the primary school.

London: David Fulton.

Darling-Hammond, L., Amrein-Beardsley, A., Haertel, E., & Rothstein, J. (2012). Evaluating teacher evaluation.

Phi Delta Kappan, 93

(6), 8-15.

Donaldson, M..L., & Donaldson, G.A. (2012).

Strengthening Teacher Evaluation: What District Leaders Can Do. To cultivate high-quality

teaching, school leaders need to focus on the human side of teacher assessment. Educational Leadership, May, 78-82.

Education Review Office (2013).

Board Employment Responsibilities: Linking Charter Targets to Appraisal in Primary Schools (February

2013), http://www.ero.govt.nz/National-Reports/Board-Employment-Responsibilities-Linking-Charter-Targets-to-Appraisal-in-Primary-

Schools-February-2013/Overview

Forrester, G. (2011). Performance management in education: milestone or millstone?

Management in Education 25

(1), 5

–9. DOI:

10.1177/0892020610383902

Hislop, D. (2012). Is performance pay just over the horizon?

Education Today, 2

, 1.

Humphrey, D.C., Gallagher, H.A., Yee, K.M., Gross, K., Campbell, A.Z., Cassidy, L.J., & Mitchell, N.M. (2012). Teacher Incentive Fund: First

Implementation Report. U.S. Department of Education Office of Planning, Evaluation and Policy Development Policy and Program Studies

Service.

Lissitz, R.W. (2012). The Evaluation of Teacher and School Effectiveness Using Growth Models and Value Added Modeling:

Hope Versus Reality. AERA Presentation, Vancouver, May.

Looney, J. (2011 ) Developing High-Quality Teachers: Teacher evaluation for improvement European Journal of Education ,

46

(4), 440-455.

Mather, K. & Siefert, R. (2011). Teacher, lecturer or labourer? Performance management issues in education.

26-31. DOI: 10.1177/0892020610383902

Management in Education 25

(1),

Moreland, J. (2011). Managing performance for effective classrooms.

Management in Education 25

(1), 21-25. DOI: 10.1177/0892020610383902

Perillo, S. (2006). Practice enhancement: Optimising teaching performance in schools.

International Journal of Educational Management, 20

(5),

365-379.

Piggot-Irvine, E. (2003). Key features of appraisal effectiveness.

Piggot-Irvine, E. & Cardno, C. (2005).

International Journal of Educational Management, 17

(4), 170-178.

Appraising performance productively: Integrating accountability and development.

Auckland: Eversleigh

Publishing.

Piggot-Irvine, E. (2010). One school’s approach to overcoming resistance and improving appraisal: Organizational learning in action.

Educational Management Administration & Leadership 3

8(2) 229

–245.

Piggot-Irvine, E. (2010). Confronting evaluation blindness: Evidence of influence of action science based feedback.

Evaluation

, 31(3), 314-325, DOI: 10.1177/1098214010369251.

American Journal of

Piggot-Irvine, E. (2012). Creating authentic collaboration: A central feature of effectiveness. In O. Zuber-Skerritt,

development in a turbulent world (pp.89-107).

Bingley, UK: Emerald.

Action research for sustainable

Sinemma, C.E.L. & Robinson, V.M.J. (2007). The leadership of teaching and learning: Implications for teacher evaluation.

in Schools, 6

, 319-343.

Leadership and Policy

Download