Educator Effectiveness:
Instruction and Assessment
Linda Barker,
Director of Teaching and Learning
Goal for today…..
Provide an overview of how quality standards,
instruction, assessment fit into a system for
improving practices of educators and student
learning results for students.
An invitation…
Requirements: SB 191
State law requirements:
• Conduct performance evaluations for all teachers and
principals at least once each school year.
• Base at least half of each teacher’s and principal’s evaluation
on multiple measures of students’ academic growth (CSAP, as
appropriate, plus other growth data).
• Requires all teachers and principals in one of the
performance standards, “highly effective”, “effective”,
“partially effective”, or “ineffective”.
Requirements cont…
State law requirements:
• Award non-probationary status to teachers with three
consecutive years of “effective” performance and remove it for
those who are not “effective” for two consecutive years.
• Consider factors such as student mobility and the numbers of
students with disabilities or at risk of failing school.
• Require mutual consent of teachers and principals to teacher
assignments.
• Factor in teacher effectiveness before seniority when
considering district-level layoffs.
The WHY….Purpose of Evaluation
According to the rules for administration of a state system to
evaluate the effectiveness of licensed personnel, the basic
purposes of this system are:
1.
To ensure that all licensed personnel are evaluated using
multiple, fair, transparent, timely, rigorous and valid
methods, 50 percent of which is determined by the
academic growth of their students and 50% on practice
1.
To ensure that all licensed personnel receive adequate
feedback and professional development support to
provide them a meaningful opportunity to improve their
effectiveness
2.
To ensure that all licensed personnel are provided the
means to share effective practices with other educators
throughout the state
Key Priorities of the System:
1.
Data should inform decisions, but human judgment
will always be an essential component.
2.
The implementation and assessment of the evaluation
system must embody continuous improvement.
3.
The purpose of the system is to provide meaningful
and credible feedback that improves performance.
4.
The development and implementation of educator
evaluation systems must continue to involve all
stakeholders in a collaborative process.
5.
Educator evaluations must take place within a larger
system that is aligned and supportive.
Components of an Effective Evaluation System
1. Evaluation
System Goals
2. Stakeholder
Investment and
Communication
Plan
3. Selecting
Measures
4. System
Structure
5. Evaluators
6. Data Integrity
7. Using Results
& Professional
Development
8. System
Evaluation
1.
Training
A Process, not an Event!
9.
Goal-Setting
and
Performance
Planning
2.
Annual
Orientation
Include any new
district or school
decisions for
measuring
Student Learning
3.
SelfAssessment
Evaluation Cycle
Principal/Assistant
Principals and Teachers
8.
Final Ratings
Review and
finalize
compiled results
of measures of
student learning
4.
Review of
Annual Goals
and
Performance
Plan
Train: Prior to the
Prior to the beginning
beginning
of
School.
May
15
End
End
of
May
September.
ofSemester
15
June
of
Spring
Mid-June
May
Orient: Within the first
week of School.
7.
End-of-Year
Review
5.
Mid-Year
Review
6.
Evaluator
Assessment
Confirm
measures used to
determine
Student Learning
– Finalize
baseline
information
Review available
information
from measures
to determine if
students are on
track
Teacher Evaluation Framework
Definition of Teacher Effectiveness
Effective Teachers in the state of Colorado have the
knowledge, skills, and commitments needed to provide
excellent and equitable learning opportunities and
growth for all students. They strive to support growth
and development, close achievement gaps and to
prepare diverse student populations for postsecondary
and workforce success. Effective Teachers facilitate
mastery of content and skill development, and employ
and adjust evidence-based strategies and approaches for
students who are not achieving mastery and students who need acceleration.
They also develop in students the skills, interests and abilities necessary to
be lifelong learners, as well as for democratic and civic participation.
Effective Teachers communicate high expectations to students and their
families and utilize diverse strategies to engage them in a mutually
supportive teaching and learning environment. Because effective Teachers
understand that the work of ensuring meaningful learning opportunities for
all students cannot happen in isolation, they engage in collaboration,
continuous reflection, on-going learning and leadership within the profession.
STATE COUNCIL FOR EDUCATOR EFFECTIVENESS
Framework for System to Evaluate Teachers
Definition of Teacher Effectiveness
Quality Standards
I. Know Content
II. Establish
Environment
III. Facilitate
Learning
50% Professional Practice Standards
Observations of
Teaching
Other Measures
Aligned with
CDE Guidelines
IV. Reflect on
Practice
V. Demonstrate
Leadership
VI. Student Growth
50% Student Growth Measures
State
Other Assessments Other Measures
Summative
for Non-tested
Aligned Assessments
Areas
CDE Guidelines
Match of test to teaching assignments
Weighting: How Much Does
Each Standard Count Towards
Overall Performance?
Weighting:
Scoring Framework: How Do Measures of Quality Standards
Result in a Determination of Individual Performance?
Performance Standards
Ineffective
Partially Effective
Effective
Appeals Process
Highly Effective
Understanding the Scoring “Business” Rule
Look for the first unchecked professional practice.
Move one column back to identify the rating for the element.
BEFORE
AFTER
Revised Teacher Rubric
Redundancies have been eliminated.
16
Rubric Structure and Rating Level Focus
The focus of the Basic
rating level is the
educator whose
performance does not
meet state performance
standards and who is
not achieving at
expected levels.
The focus of Partially Proficient
and Proficient levels is what
educators do on a day-to-day basis
to achieve state performance
standards and assure that students
are achieving at expected levels.
The focus of Accomplished and
Exemplary ratings shifts to the
outcomes of the educator’s practices,
including expectations for staff, students,
parents and community members, as a
result of practices exhibited under rating
levels 2 and 3.
Teacher Evaluations
I. Mastery of content
II. Establish learning
environment
III. Facilitate learning
IV. Reflect on practice
V. Demonstrate
leadership
Measured using multiple measures
on multiple occasions, including: (1)
observations; and (2) at least one of
the following: student perception
measures, where appropriate and
feasible, peer feedback, feedback
from parents or guardians, or review
of teacher lesson plans or student
work samples. May include
additional measures.
50%
Professional
Practice
It’s All About the Practice
CDE tool for Observable Practices…
The Teacher
Element a:
 Uses lesson plans that reflect:
 Daily review and revision.
 Instructional objectives appropriate for
students.
 Explicit connections to specific learning
objectives and approved curriculum
 Implements lesson plans based on
 Student needs
 Colorado Academic Standards.
 District’s plan of instruction.
 Stated learning objectives.
 Collaborates with other school staff to
vertically and horizontally align,
articulate, and deliver the approved
curriculum.
Students:
 Interact with the rigorous and
challenging content.
 Perform at a level consistent
with or above expectations.
 Discuss strengths and next
steps regarding their learning
with their teachers.
Teacher Self-Assessment Ratings
Distribution
Teacher Self-Assessment Ratings-Standard 3plan and deliver effective instruction.
Theory into Practice…are rubrics real?
Portscheller,
Schechty,
Marzano
Student Learning Outcomes
Here’s a simple truth that is easy to forget.
School is not about grading. School is about…
Learning
STATE COUNCIL FOR EDUCATOR EFFECTIVENESS
Framework for System to Evaluate Teachers
Definition of Teacher Effectiveness
Quality Standards
I. Know Content
II. Establish
Environment
III. Facilitate
Learning
IV. Reflect on
Practice
50% Professional Practice Standards
Observations of
Teaching
Other Measures
Aligned with
CDE Guidelines
V. Demonstrate
Leadership
VI. Student Growth
50% Student Growth Measures
Weighting: How Much Does
Each Standard Count Towards
Overall Performance?
State
Other Assessments Other Measures
Summative
for Non-tested
Aligned Assessments
Areas
CDE Guidelines
Match of test to teaching assignments
Weighting:
Scoring Framework: How Do Measures of Quality Standards
Result in a Determination of Individual Performance?
Performance Standards
Ineffective
Partially Effective
Effective
Appeals Process
Highly Effective
Teacher Evaluations
VI. Responsibility for student
academic growth
Refers to outcomes on a
measure that are
attributed to an
individual licensed
person,
e.g. DRA2 growth
measures for a 1st Grade
Teacher’s students
Refers to outcomes on a
measure are attributed
to two or more licensed
personnel,
e.g. 10th gr. Math TCAP –
All Secondary math
teachers in school
50% Student
Learning
Outcomes
Evaluated using the following:
(1) a measure of individuallyattributed growth,
(2) a measure of collectivelyattributed growth;
(3) when available, statewide
summative assessment results;
and
(4) for subjects with statewide
summative assessment results
available in two consecutive
grades, results from the
Colorado Growth Model.
It’s All About the Questions?
Student Learning Outcomes
How do measure student learning outcomes in
your district/school?
Collective vs. Individual Attribution
Do you understand what % your district (1338) has
assigned for collective or individual attribution in
your evaluation system.
Categorizing Teachers
Example:
Colorado Growth
Model (CGM)
Reading, Writing,
Math
(Gr. 4-10)
2.
State Summative
Reading, Writing,
Math (Gr. 3 & 1st Yr. Gr. 410)
Science
(Gr. 5, 8, 10)
No Summative/CGM
All Other Teachers
70%
Social Studies
(Gr. 4, 7, HS)
ACT (HS)
WIDA ACCESS (ELL)
What are the pros and cons of creating groups of like teachers based
on the types of measures that must be included in their body of
evidence?
Based on what is expected in law, what categories of teachers are
possible?
Student Learning Outcomes
Select and weight multiple measures of student
learning to be included in educator evaluations.
What can we do as a district to build comparable bodies of
evidence in like groups of teachers?
Questions to consider :
 What assessments must we include?
 How much influence will they have on the evaluation?
 Student Learning Outcomes Tool(SLOT)
Using Local Assessments in Evaluation
 Districts, BOCES and schools may decide after
completing the Assessment Inventory that a locally
created assessment would likely be included as a
measure in evaluation. Which ones?
 Districts and BOCES can use the Assessment Review
Tool to show they are taking steps to ensure that the
locally created measurements they select are fair, valid,
and reliable.
Categorizing, Weighting, and Selecting
Measures Reflection
Decision Points
 How will our district identify teacher roles and the
types of assessments that exist?
 Who will participate in the process?
 What do we need to know in order to make good
decisions?
 What resources (internal and external) do we
have/need?
 How will we know if we have made good
decisions?
Setting Student Learning Targets
What are Student Learning Targets?
 Method of measuring student growth, progress or mastery
of the standards
 Long-term academic goal set by educators for students
 Based on available prior student learning data and information
 Set at the beginning of the year/course by teachers in
collaboration with their supervisor or evaluator
 Can be shared by teachers of the same course/grade level
or individually
Student Learning Target
Infrastructure
Curriculum
Colorado
Academic
Standards
Assessments
Student
Learning
Target
Classroom
(Math) Standard 6
13%
NWEA Standard 6
13%
DIEBELS Standard 6
12%
TCAP Standard 6
12%
Performance
Tool Standards
1-5
50%
Practices
50.0%
SPF
37.5%
Individual
12.5%
NWEA, targets, %, grades,
SLOs,
District & School Performance Frameworks
Through the Colorado Educational Accountability Act of
2009 (SB09-163)…
 CDE annually evaluates districts and schools based on student
performance outcomes.
 All districts receive a District Performance Framework (DPF).
This determines their accreditation rating.
 All schools receive a School Performance Framework (SPF).
This determines their school plan types.
 Provide a common framework through which to understand
performance and focus improvement efforts.
Accreditation & Plan Types
 Accreditation designations:
 Accredited with Distinction (10%)
 Accredited (50%)
 Accredited with Improvement Plan (25%)
 Accredited with Priority Improvement Plan (10%)
 Accredited with Turnaround Plan (5%)
 School plan types:
 Performance Plan (60%)
 Improvement Plan (25%)
 Priority Improvement Plan (10%)
 Turnaround Plan (5%)
Unified Improvement Planning
 Through the Colorado Educational Accountability Act
of 2009 (SB09-163)…
 All schools and districts must annually develop and submit an
improvement plan. The plan streamlines federal and state accountability
systems and reporting requirements.
 Plans must include:
 Data Analysis: trends, performance challenges, root causes
 Action Planning: targets, interim measures, major improvement strategies,
action steps, implementation benchmarks
 Aligned with ESEA (Titles I, IIA, III), State Graduation
Completion Plans and various grant requirements.
Improvement Planning Process
Preparing
to Plan
Gather and
Organize Data
Section III:
Data
Analysis
and Data
Narrative
Review
Current
Performance
Section IV:
Target Setting
Describe
Significant
Trends
Ongoing:
Progress
Monitoring
Section IV:
Action Planning
Prioritize
Performance
Challenges
Identify
Root
Causes
Set
Performance
Targets
Identify Major
Improvement
Strategies
Identify
Interim
Measures
Identify
Implementation
Benchmarks
Assessment and
Instruction/Practice
Dr. Harvey Silver’s work on
connections…9 years—”formative
assessment leads to dramatic gains in
achievement, one of the most powerful
tools a teacher can use”.
Assessment Inventory
ASSESSMENTS
What
of student learning do
you currently use in your classroom, caseload practice
or school?

Standardized Tests

Textbook-based Tests

Teacher-developed Unit Tests: e.g. quadratic equations,
Catcher in the Rye, photosynthesis.

Mid-Year & Finals
 Common across subject (all students in the
grade/subject take the same test)
 Common across schools (all students in the
grade/subject across all schools take the same test)

Performance Assessment: lab experiment, speaking a
second language, research project, serving a volleyball,
solving a math problem with written explanation.

Writing Tasks: compositions, essays, reports

Portfolios: collection of student work over time

Exhibitions: oral report, art show, musical performance,
DECA competition, science fair, capstone projects.
 Use of assessment
diagnostic
 interim/benchmark
 summative
 Non-cognitive
 Value Judgment
 Do we value the
assessment(s)?
Why?

Think about your
assessments/purpose/audience:
Feedback/Instruction
… FOR learning
Data/Trends (after T and L)
….OF learning
 Exit tickets
 TCAP
 Sponge pads
 SAT
 Work folders
 ACT
 Writing logs
 Previous year’s grades
 Discussion records
 District tests
 Journals
 Final exams
 Writing samples
 Chapter quizzes and tests
All grades, all levels…..
Your job is to
uncover
understanding,
not cover content.
Assessments
Science and Social Studies
Summative Assessments
Colorado is piloting new science and social studies
assessments. These assessments will be computer-based and will
be designed to measure the CAS. These assessments will be
administered in 2014.
The science assessments will be administered in grades 5, 8, and
12
and ……
social studies assessments will be administered in grades 4, 7, and
12th grade.
Practice testing environments, also referred to as ePATs (electronic
Practice Assessment Tools), that help to familiarize students with
the testing environment can be accessed at:
http://www.pearsonaccess.com.
PARCC in 2014-15
Partnership for Assessment of Readiness of College
and Careers
PARCC’s next-generation assessment system will
provide students, educators, and the public with the
tools needed to identify whether students — from
grade 3 through 11th grade in math and language
arts/literacy— are on track for postsecondary
success.
How will the new assessments be
different from CSAP/TCAP?
 Summative assessments aligned to College and Career




Readiness Standards
Computer adaptive
Might not be a set “test window”
Results will show achievement and growth
Question types:
1. Selected Response
2. Constructed Response
3. Extended Response
4. Performance –Based Tasks
5. Technology-Enhanced
6. Technology-Enabled
Grade 3 Technology-Enhanced
Constructed-Response Item
Drag the words from the word box into the correct
locations on the graphic to show the life cycle of a
butterfly as described in “How Animals Live.”
Words:
Pupa
Adult
Egg
Larva
Grade 7 Analytical Constructed Response
You have read three texts describing Amelia Earhart. All three include the claim that
Earhart was a brave, courageous person. The three texts are:
•
“Biography of Amelia Earhart”
•
“Earhart's Final Resting Place Believed Found”
•
“Amelia Earhart’s Life and Disappearance”
Consider the argument each author uses to demonstrate Earhart’s bravery.
Write an essay that analyzes the strength of the arguments about Earhart’s bravery
in at least two of the texts. Remember to use textual evidence to support your ideas.
CCSS-use of evidence, summary of text, complex texts, writing to explain/inform, writing
coherently, drawing evidence from texts, grammar and conventions, comparisons of authors,
relevant info from multiple sources.
Assessments
 How do these questions differ from the
questions we currently ask on TCAP?
 Interdisciplinary, online, not one choice,
multiple possible answers, defend and justify
their answers, mental manipulation, inference,
transfer….not your mom/dad or even older
brother test!
 What are the implications for classroom
instruction?
Still …
Specialized Service Professionals
 Rubrics are currently being piloted for the following
Specialized Service Professional groups:
Audiologists
Counselors
Nurses
Occupational Therapists
Orientation and Mobility
Specialists
Physical Therapists
School Psychologists
School Social Workers
Speech Language Pathologists
 SSP Pilot Year 2013-2014
 Pilot sites will pilot the evaluation system for those SSP groups
they have selected
 SSP Implementation and Validation Study Year 2014-2015
 All districts will evaluate their SSPs under SB 191 requirements
 CDE will conduct a validation study on the evaluation rubrics
57
Districts decide…
measures
Standards I-V: use observation plus at least one other
Standard VI: select multiple measures appropriate to
method
teaching assignment
weights
On each Standard I-V districts may weight priority
standards more
Standard VI must count for at least 50% of total score
data collection procedures
Standards I-V: Must occur with enough frequency to create a
credible body of evidence
Standard VI: Must occur with enough frequency to
create a credible body of evidence
aggregate measures
Aggregate professional practice scores into a single
score on Quality Standards I-V
Aggregate student growth measures into a single score
on Quality Standard VI
District uses State Scoring Framework Matrix to determine Performance Standard
Timeline of Implementation
Year One 201112 Development
and Beta Testing
• CDE ACTIVITIES
• Develop State Model
Systems for teachers
and principals
• Beta-testing of rubrics
and tools
• Develop technical
guidelines on Prof
Practices and Student
Growth
• Provide differentiated
support for districts
• Populate and launch
online Resource Bank
• Develop state data
collection and
monitoring system
• Develop tools for
district
implementation of
system
Year Two 2012-13
Pilot and Rollout
• CDE ACTIVITIES
• Usability study of
rubrics
• Support pilot
districts through
resources, training,
tools, etc.
• Convene pilot
districts to share
lessons learned
• Analyze pilot district
data and make
adjustments as
needed
• Train ALL non-pilot
districts that are
using the state model
• Make
Recommendations
on Specialized
Service Professionals
(SSP)to State Board
of Education (SBE)
Year Three 201314 Pilot and
Rollout
• CDE ACTIVITIES
• Statewide TA on
rollout of
teacher/principal
systems
• Develop evaluation
system for
Specialized Service
Professionals (SSP)
• Support ALL districts
through resources,
trainings, tools, etc.
• Convene pilot
districts to share
lessons learned
• Analyze state data
and make
adjustments to the
system as needed
• Validate teacher and
principal rubrics
• Dev. criteria for eval
training courses for
approval
Year Four 201415
Full
Statewide
Implementation
• CDE ACTIVITIES
• Finalize statewide
implementation of
teacher/principal
systems
• Statewide roll out and
validation of SSP system
• Continue support to
districts via resources
and training
• Ensure there are
evaluator training
courses throughout the
state
• Analyze data and make
adjustments as needed
• Make recommendations
to SBE this year and all
following years for
Continuous
Improvement
Questions?
Resources?
http://www.cde.state.co.us/educatoreffectiveness/
statemodelevaluationsystem
 Tools, webinars, fact sheets, FAQs, videos……
 User guide-all the answers!
http://www.cde.state.co.us/educatoreffectiveness/coloradolegacy
foundation