Educator Evaluation:
Research-Informed Practice
Dr. Don Beaudette
Rachel Bradshaw
Richard Fournier
Elizabeth Nolan
Educator Evaluation: Research-Informed Practice
September 18, 2014
Agenda
Boston University Slideshow Title Goes Here
1. Student Input in Teacher Evaluation (Rachel Bradshaw)
2. Instructional Leadership in the Context of Teacher
Evaluation (Elizabeth Nolan)
3. The Evaluation of Special Educators (Richard Fournier)
Educator Evaluation: Research-Informed Practice
Boston University Slideshow Title Goes Here
Part 1
STUDENT INPUT IN TEACHER
EVALUATION
September 18, 2014
Educator Evaluation: Research-Informed Practice
What is worth knowing?
Slideshow Title Goes Here
1.Boston
In University
this class,
we learn to correct our mistakes.
2. This teacher is nice to me when I need help.
3. We learn a lot in this class.
4. I look forward to coming to this class.
5. In this class, we stay busy and do not waste time.
6. This teacher pushes us to use our thinking skills.
7. Students do what this teacher wants them to do.
8. This teacher makes learning interesting.
9. This teacher wants us to share our ideas.
10. This is a good teacher.
September 18, 2014
Educator Evaluation: Research-Informed Practice
September 18, 2014
Are student perceptual data trustworthy?
Boston University Slideshow Title Goes Here
 Yes …with a well-designed survey
 Witnesses to behaviors, not judges of skill
 Redundant, research-based items
 Reliability
 Inter-rater: ten or more students
 Test-retest: two or more years?
 Validity
 Convergent: observations, self-reports
 Predictive: test scores
Educator Evaluation: Research-Informed Practice
September 18, 2014
What roles can student surveys play among
multiple measures?
Boston University Slideshow Title Goes Here
 Monitoring implementation processes
 Alignment with Danielson framework
 Adding nuance to evaluation
 Complementing, not just verifying
 Providing “fast and frequent feedback”
 Multiple administrations per year
 Engaging and empowering students…
 …and teachers!
Educator Evaluation: Research-Informed Practice
What are best practices for
implementation?
Boston University Slideshow Title Goes Here
 Understanding and buy-in
 For data as well as for people
 Equity and protection for students
 Translations, accommodations, proctors
 Norming by age, subject, etc.
 National data sets, anti-bias algorithms
 Focus on improvement
 Actionable feedback, professional development
September 18, 2014
Educator Evaluation: Research-Informed Practice
September 18, 2014
Boston University Slideshow Title Goes Here
Part 2
INSTRUCTIONAL LEADERSHIP IN THE
CONTEXT OF TEACHER EVALUATION
Educator Evaluation: Research-Informed Practice
Boston University Slideshow Title Goes Here
September 18, 2014
Educator Evaluation: Research-Informed Practice
September 18, 2014
Which do you emphasize more in your
district?
Boston University Slideshow Title Goes Here
Educator Evaluation: Research-Informed Practice
September 18, 2014
What is the purpose of teacher evaluation?
Boston University Slideshow Title Goes Here
Accountability
 State/District Standards
Improvement
 Formative Evaluations
 Retention
 Feedback
 Dismissal
 Professional
Development
Educator Evaluation: Research-Informed Practice
September 18, 2014
What does the research say about observation
protocols
as a tool for improvement?
Boston University Slideshow Title Goes Here
 Error Marzano (2012)
 Measurement
 Sampling
 3 Types of Lessons
 Introductory
 Practice and Understanding
 Application
 Assumptions
 Principals are content experts
 Generic Instruments
Educator Evaluation: Research-Informed Practice
September 18, 2014
How can observations be more accurate and promote teacher
improvement?
Boston University Slideshow Title Goes Here







“Differentiating” the number of observations
Looking for content expertise
Conferring with teachers
Strategically planning observations
Conducting walk-throughs
Videotaping lessons
Peer Assistance and Review (PAR)
Educator Evaluation: Research-Informed Practice
September 18, 2014
How can a principal make this all work?
 Use Technology
 TeachPoint
 Evernote
 Excel
 Limit the focus
 Ask teachers to do some homework
 Pre-conference worksheets
 Data
 Devote a day
 Setting targets and goals
Boston University Slideshow Title Goes Here
Educator Evaluation: Research-Informed Practice
September 18, 2014
What is instructional leadership in the
context of teacher evaluation?
Boston University Slideshow Title Goes Here
 Traditional Idea
 Focus on curriculum and instruction
 Limited if any improvements in student growth
 Contemporary Idea
 Focus on organizational management
 Growth in student achievement
Educator Evaluation: Research-Informed Practice
September 18, 2014
How can central office administrators support
principals
as instructional leaders?
Boston University Slideshow Title Goes Here
 Training
 Ongoing professional development
 Time with staff to focus on the evaluation
 Instructional coaching
 Credentialing
 Provide opportunities for principal feedback on the
evaluation process
 Allow for flexibility with the number of observations
 Allocate funding for support personnel to assist
 Provide technology to facilitate observations
Evaluating
Special Education Teachers:
Current Issues & Recommendations
Richard Fournier
Doctoral Fellow
rfournie@bu.edu
18
Introduction: Why Special Education?
Boston University Slideshow Title Goes Here
Evaluating Special Education Teachers:
Major Issues
Boston University Slideshow Title Goes Here
 Special education has unique characteristics
evaluation models rarely consider
• These features challenge current models of
evaluating teacher effectiveness:
① Diversity in population of students
with disabilities
② Intensive, individualized instruction
found in SPED
Evaluating Special Education Teachers:
Major Issues
Boston University Slideshow Title Goes Here
 Problems
 Current mixed-method measures (e.g., VAM) are not easily
transferrable to special education
 Current models tend to be “one-size-fits-all”
 Current teacher observations problematic
What does the research say?
Boston University Slideshow Title Goes Here
Rigorous research is limited, but
growing
Little understanding of what
models work or of alternative
solutions
What does the research say?
Observation of Special Education Teachers
Boston University Slideshow Title Goes Here
Examples:
•Danielson’s Framework for Teaching (FFT)
• Meant to be comprehensive of all aspects of one’s teaching
•Classroom Assessment Scoring System (CLASS)
• Focuses on teacher-student interactions
Recent Findings:
•
•
•
•
•
observations scores tend to be unstable
Raters tend to remain high or low in ratings
Consistency with observation systems
Not strongly correlated with VAM scores
BIG ISSUE: How do we deal with issues of fairness?
Dr. Nathan Jones (2013), Presentation with Office of Special Education Programs (OSEP)
What does the research say?
Observation of Special Education Teachers
Boston University Slideshow Title Goes Here
Practical, Common Issues:
• Evaluators unfamiliar with environment
• Differences in ideas and definitions of effective teaching
• Roles & Responsibilities not captured in protocol
Dr. Nathan Jones (2013), Presentation with Office of Special Education Programs (OSEP)
What does the research say?
Observation of Special Education Teachers
Boston University Slideshow Title Goes Here
Unique roles and
responsibilities
Johnson, E., & Semmelroth, C. L. (2014).
What does the research say?
Bostonresearch
University Slideshow
Title Goes Here
New
emerging…
What doesn’t work?
What can be done to improve existing models?
Practical Recommendations for Observations:
Ensure that special education teachers are familiar with
checklist items OR rubric (e.g., FFT)
Have evaluators and teachers select specific domains to
work on
Encourage and allow time for teachers to explain their
lesson plans or actions during evaluation
Video exemplars for administrators
Provide extra training for administrators evaluating
special education teachers
What does the research say?
Observation Tool in Development
Boston University Slideshow Title Goes Here
RESET
Recognizing Effective Special Education Teachers
What will it include?
① Video component
② Acknowledge unique roles and settings
③ Scoring on specific instruction
strategies
④ Collecting student growth data (still in
development)
Johnson, E., & Semmelroth, C. L. (2014).
What does the research say?
Boston University Slideshow Title Goes Here
PEER Observation/Review of Special Education Teachers
Recommendation to improve observations for special education teachers
Inform general education observers/reviewers
Document peer observer notes and review
Peer observer and teacher identify specific areas
Encourage teachers to take feedback seriously
What does the research say?
Boston University Slideshow Title Goes Here
Portfolios for Special Education Teachers
May want to consider e-portfolio system
Administrators should review often
(with teachers)
Provide model portfolios & handbook
Enlist special education
teacher/administrator to review portfolios
with main reviewer
Educator Evaluation: Research-Informed Practice
September 18, 2014
What does the research say?
Boston University Slideshow Title Goes Here
Dynamic Instructional Practices Portfolio Tool
MyiLOGS My Instructional Learning Opportunities Guidance System
Educator Evaluation: Research-Informed Practice
September 18, 2014
Lastly . . .
Boston University Slideshow Title Goes Here
 Recognize professionalism
 Incorporate Research
 Where?
Regional Educational Laboratory (Northeast & Islands)
www.relnei.org
Educator Evaluation: Research-Informed Practice
September 18, 2014
Free Research Resources!
Boston University Slideshow Title Goes Here
Ask A REL is a free reference desk service providing brief
responses to your education-related questions.
Free Research Resources!
Boston University Slideshow Title Goes Here
• Education Resources Information Center (ERIC)
• Free access to > 1 million records of journal articles
• Often has full text!! (http://www.eric.ed.gov/)
• What Works Clearinghouse (WWC)
• Free, easily accessible databases and reports
• Provides high quality reviews of educational
interventions (programs, products, practices, policies)
• Doing What Works (DWW)
• Free & Helps educators identify and use effective teaching
practices
• Provides examples of ways educators might apply research