DEVELOPING DEEPER AND
MORE MEANINGFUL
EVALUATION SYSTEMS
Funded by a grant through
the McCormick Foundation
Lisa Hood, Center for the Study of Education Policy
Teacher Evaluation in the Early Grades:
Developing a PreK-3 Support Services for
Danielson Framework for Teaching
Danielson Early Supports Project Staff
Lisa Hood, lhood@ilstu.edu
Deb Kasperski, dkasper@ilstu.edu
Erika Hunt, elhunt@ilstu.edu
Pam Rosa, pam.rosa@cecillinois.org
Website
http://leadershiplinc.illinoisstate.edu/
District Level – Teacher Evaluation
in the Early Grades



Danielson Framework for Teaching has been
validated for Grades 4 and above, yet many districts
were planning to use the Danielson FfT for early
childhood teachers (PreK-grade 3)
Is this a valid and reliable tool for PreK-3 teachers?
What tiered assessments are appropriate for PreK-3
classrooms?
Developing Early Learning Supports for
Danielson Framework for Teaching (FfT)







Fall 2012:Review and adaptation by PreK-3 administrators,
teachers, and stakeholders
 Critical Attributes and Examples
Winter 2012: NBCT’s developed examples and coded PreK-3
videos using adapted Framework
Winter 2013: Training developed for pilot principal and teachers
March 15, 2013: Feedback from early childhood stakeholders
April 26, 2013: Second round of feedback from early childhood
stakeholders
June 2013: National feedback of framework supports solicited
at NAEYC conference, San Francisco
Fall 2013: Pilot of framework supports and training
Piloting the 2013 FfT with Early
Learning Supports


Pilot will occur in seven selected schools throughout the
region (Northern, Central, and Southern Illinois) in fall
2013.
Prior to pilot, selected principals and teachers will
undergo a three day pilot training that will utilize three
inter-connected modules.
Module 1: Understanding and using the 2013 Danielson FfT
in the context of early learning (PreK-3rd grade)




Present the need for piloting the 2013 FfT Early Learning
supports to enhance early learning teaching practice growth
opportunities
Identify the structure/priorities of the 2013 FfT
Domains/Components and Critical Attributes
Understand shifts in professional practice for different levels of
performance and the link to the 2013 FfT Framework
Components/Critical Attributes
Explore and connect to Early Learning Grade Band examples
at four different levels of practice
Module 2: Applying the 2013 Danielson FfT instrument
through observation-based evidence collection in early
learning teaching settings



Understand observation-based data collection within the
Plan-Teach-Reflect-Apply (PTRA) Professional Improvement
Cycle for supporting and improving teaching practice
Develop/enhance observation process skills for collecting,
sorting, and scoring evidence focused upon early learning
Classroom Environment and Instruction
Determine Master Scores for early learning Classroom
Environment and Instruction teaching practices using 2013
FfT Components and Critical Attributes
Module 3: Applying the 2013 Danielson Framework for
Teaching instrument through conversation-based evidence
collection in early learning teaching settings




Understand data collection and artifacts of practice and student
learning within Professional Improvement Cycle
Enhance skills focused upon early learning Planning and
Preparation practices, review of Classroom Environment and
Instruction, and Responsibilities
Determine Master Ratings for Planning/Preparation and
Responsibilities using FfT Components and Critical Attributes and
protocols to confirm inter-rater reliability
Extend knowledge of professional learning-focused coaching
language, conversations, and protocols of practice to support
planning and reflection conversations
New Proposals


February 2013: Submitted a proposal to validate
the 2013 Danielson Framework for Teaching for
PreK-3rd grade teachers
April 2013: Submitting a proposal to validate the
training and professional development modules for
principals and teachers associated with the early
learning supports for the 2013 Danielson FfT.
7/25/11
Purpose of Proposed
Validation Study



To ensure that FfT provides accurate and reliable data that
PreK-3rd grade teachers and their supervisors can use to
identify strengths and weaknesses
To help teachers and supervisors use the data from the FfT to
identify appropriate professional development mechanisms to
promote teachers’ growth.
To utilize data from the early learning Framework’s observation
along with other with student growth data so that teachers who
are observed to be “Unsatisfactory, Needs Improvement,
Proficient, or Excellent” have appropriate student growth that
correlates with the observation ratings
Other Benefits of Proposed
Validation Study



Construct a student/parent survey
Explore other student growth measures
Increase principals’ understanding of different teaching
pedagogies related to early childhood education
Purpose of Proposed Professional
Development Project



Develop and validate professional development training
and supplemental materials to support Danielson FfT for
PreK-3rd grade teachers.
To use the data from validation study to identify
appropriate professional development mechanisms to
promote teachers’ growth and principals’ understanding of
early childhood education (PreK-grade 3).
Training geared to principals, other persons evaluating
teachers, the PreK-3rd grade teachers
Group Activity


Question One: What do evaluators of PreK-grade
3 teachers need to know about early learning
(PreK-grade 3) classrooms?
Question Two: What do principals need to know in
general about doing effective evaluations that will
grow teacher’s practice?
Building Evaluation Capacity Within P-12
Principal Programs and Partners
Approval of new Principal Endorsement
Programs – a beginning not an end

Annual reports required by Illinois Educator
Licensure Board
 program
data (e.g., number of faculty, number of
graduates, etc.)
 formative data that is internal to the programs and
varies by institution

2011 - Recommendations by P-20 Council
Committee on Teacher and Leadership Effectiveness
for Principal Preparation Performance Indicators\
 Tied
to new P-20 Data Longitudinal System
Principal Preparation Program Performance
Indicators
Principal Performance Indicators Recommended by P-20 Council Teacher and
Leadership Effectiveness Committee (2011)
1.
Percentage of graduates that pass the required state principal endorsement certification exam
Note: This is a competency based assessment that will be administered to all candidates
2.
Percentage of program graduates that obtain principal or assistant principal positions within 1, 3 and
5 years of graduation
3.
Performance evaluations of program graduates using the four category rating system once placed as
principals and assistant principals in Illinois
Note: This data will be available after implementation of the new statewide requirements in the 2012-13
school year.
4.
Data and indicators of student growth within 1, 3 and 5 years of placement as an assistant principal
or principal
Note: According to Illinois requirements, this measure will be incorporated into performance evaluation
ratings of Illinois principals and assistant principals.
5.
Results of learning climate surveys in schools where principals and assistant principals are placed
within 1, 3 and 5 years of graduation
Note: A state standardized learning climate survey will be developed
6.
Percentage of program graduates retained as principals, assistant principals, superintendents or
district leadership roles within 1, 3 and 5 years of placement
Principal Preparation Program
Continuous Improvement Process Project


Project Staff

Lisa Hood, Project Director

Erika Hunt, Project Staff

Alicia Haller, Evaluation Coach

Joe Pacha, Evaluation Coach

Brad White, Illinois Education Research Council, Team Consultant
Project Partners:

University of Illinois at Urbana-Champaign

Southern Illinois University at Edwardsville

North Central College

Loyola University

New Leaders for New Schools
Logic Model: Illinois Principal Preparation Program Continuous Improvement and Review Process
Outputs:
Inputs:
(What to invest)
Faculty
Time
Money
Research
Materials
Technology
Partners
Outcomes-Impact
Activities:
(What we do)
Participation:
(Who we reach)
Review Principal
Preparation Programs
(Quality Program
Assessments and
Feedback.
Principal
Preparation
Program Faculty
members and
ISBE
Support university
personnel in their
continuous
improvement processes
(Process
Improvements)
Help faculty implement
best practices for
improving programs
(Knowledge Building)
University
Partners,
Faculty, and ISBE
Faculty
Members and
ISBE
Short Term:
(What the short
term results are)
Medium Term:
(What the medium term
results are)
Improved
candidate
selection
Candidates prepared
through quality
coursework
Improved
courses and
programs
Candidates ready to
perform internship
assessments successfully
Improved
internship
Candidates prepared to
lead schools in improving
learning for all students
Improved
graduates of
programs
Long Term:
(What the
ultimate impacts
are)
Improved
learning for ALL
students
Improved
teacher practices
Improved school
leadership
Logic Model: Illinois Principal Preparation Program Continuous Improvement and Review Process
Evaluation: 1. Focus – 2. Collect Data – 3. Analyze and Interpret – 4. Report – Repeat the Cycle
Focus
1. Program Requirements
2. Internship Requirements
3. Internship Assessments
4. Coursework Requirements
5. Staffing Requirements
6. Candidate Selection
20
Collect Data
1. Data Collected for Program
Improvement
2. Analysis of the Program
Improvement Data
3. Program Changes Based on Data
Analysis
4. Summative INTERNSHIP
Assessment Data: (Number of
candidates; listing of each and all
assessments with the scoring of
each candidate on the
assessment and number of times
each candidate needed to
complete the assessment
successfully; length of the
internship for each candidate)
5. Listing of Courses and Course
Syllabi
6. Number of candidates and
Faculty (FTE)
7. Number of Candidate
Applications
8. Number of Candidates Selected
9. Number of Candidate Completers
10. Graduate’s School Data Analysis
over time
Analyze the results
1. Analysis and planning for
program improvement
based on data 1, 2, &3.
2. Analysis and
implementation for
improvement for
Internship and candidate
success based on data 4,
5, 6, 7, 8 and 9.
3. Analysis and planning for
improvements based on
data 10.
Report
1. Program Improvement
Plan
2. Internship
Improvement Plan
3. Candidate Selection
Improvement Plan
4. Coursework and
Staffing Improvement
Plan
Project Objectives





Implementation of a Program Theory Evaluation (PTE) model to
identify how policies, practices, and processes have changed as a
result of the process of redesigning;
Creation of an evaluation plan of Principal Preparation Programs;
Alignment of current data and creation of needed data sources to
continuously inform evaluation plan process;
Coaching, technical assistance, and tools to inform the evaluation
process
Collaboration with the ISBE to provide input to the annual reporting
process and requirements for Principal Preparation Programs.
Project Activities


Programs will receive technical assistance through evaluation
coaches to help build internal capacity
Programs will be provided with:
Tools to inform the evaluation process (e.g., Self-Assessment
Rubric, Sample Data Instruments)
 Technical assistance (e.g., Coaching and Program Team Meetings)
 Meeting facilitation
 Timeline planning
 Data analysis
 Reporting framework

Two Project Phases
Phase One: Initial Assessment Phase
 Phase Two: Continuous Improvement
Phase

Phase One: Initial Assessment
Phase




Beginning March 2013, evaluation coaches will convene program
faculty and partners.
Step One: Completion of a self-assessment process developed by
project staff
Step Two: Use self-assessment instrument to capture baseline
information on:
 partnerships with districts and other community members
 content and pedagogy
 assessments
 internship
Step Three: Share results with program team and develop action
plan
Phase Two: Continuous
Improvement Phase


Evaluation coaches and program faculty/partners will
begin testing the program’s theory of change and
whether it is having the intended effects on principal
candidates and new principals.
 Assessment of the program’s “continuous
improvement evaluation plan” will be conducted to
determine appropriate level of outcomes
Center staff and evaluation coaches, in partnership with
program faculty and partners, will document the process
and any on-going program modifications made.
Next Steps


Interest with funding a multi-phase statewide
evaluation of principal preparation programs
Proposed development of a framework for a
statewide evaluation of principal preparation
 Advisory
committee to assist with development of
Evaluation Framework
 Statewide forum to collect input into draft conceptual
framework
 Development of an Request for Proposal (RFP) to select
an external evaluator for the statewide evaluation
Group Activity


Question One: What are the highest priority issues
to keep in mind when developing a statewide
evaluation framework for principal preparation?
Question Two: What do you foresee will be the
benefits and challenges to doing a statewide
evaluation of principal preparation programs?
Questions?
Thank you!

Please remember to complete an evaluation form
before you leave.
 Evaluation
form includes questions on challenges and
successes of the new P-12 Principal Endorsement
programs.