STUDENT LEARNING OBJECTIVES:
An Emerging State Framework for Design
Lessons from the Teacher and Leader Evaluation Multi-State Network
January 11, 2013
DRAFT
1
Purpose of Today's Presentation
As increasing numbers of states begin reforming their systems of
educator evaluation to reflect a significant emphasis on student
performance, there is growing demand to codify sound frameworks of
Student Learning Objectives (SLOs), as an option for measuring teacher
impact on growth beyond traditional growth or value-added models.
 Several leading states are following the practice of early adopter districts and
implementing SLOs as a measure of student growth. From that work, a
framework and a concrete set of options have emerged.
 The objective of today's presentation is to provide state teams that are designing
and implementing new systems of educator evaluation with a framework for
developing an SLO process, including an analysis of the opportunities and
challenges of these measures as well as examples from early-adopter states.
 Further, it lays out a vision for how SLOs might be used in the short-term and
long-term to improve instructional practice, teacher effectiveness, and ultimately
student achievement.
2
Options for Measuring Growth in NTGS
Based on the work of several leading states and districts, there
is a concrete set of options for measuring growth in non-tested
grades and subjects (NTGS). These options, which are not
mutually exclusive, are still emerging and will need to be
refined and tightened over time to ensure quality.
Options:
 Extend assessment coverage to other subjects or grades through
additional assessments developed or selected by the state or district.
Examples include: Hillsboro ISD, Delaware, ACT, NWEA, end of course
exams, etc.
 Broaden attribution of student growth on current assessments to
additional subject areas, such as attributing growth on math assessments
to science, or more broadly to the grade or school-wide levels. Examples
include: Delaware, Indiana, Tennessee, New York
 Measure teacher progress against SLOs using district selected or teacher
generated tests, end-of-course tests, or portfolio of student work.
Examples include: DCPS, Austin ISD, New Haven, Indiana, Rhode Island,
New York
--3
Defining Student Learning Objectives
"Student learning objectives" is a broad term used to
describe a process to assess teacher impact on student
performance that involves assessment selection and goalsetting at the district, school, or teacher level.
 Notably, the term SLOs can imply a range of structures that can vary
significantly in terms of the prescriptive nature of the process and the
degree to which educators have the freedom to select individual
assessments and targets.
4
An Overview of SLOs
Recognizing that student learning is an important part of
improving educator practice and a key underpinning of assessing
teacher effectiveness, SLOs can play a critical role in new systems
of educator evaluation in many states.
 Promising practice suggests that multiple, meaningful measures of student
growth are needed for strong evaluation systems, which can be particularly
difficult for teachers who do not have data from state assessments and
traditional growth models.
 The use of SLOs in statewide systems of evaluation is a new and quickly
evolving practice, drawing from the lessons-learned and promising practices
of early-adopters. Although difficult to implement well, SLOs have the
potential to continuously improve and advance over time to ensure stronger
validity of the evaluation as well as greater impact on instructional quality.
 Recognizing that present focus must be on establishing a basic system to
meet immediate needs, states should not lose sight of the long-term vision in
which SLOs can significantly improve teaching practice and support broader
state college and career ready reforms, e.g. Common Core implementation.
5
Benefits and Challenges
SLOs offer a number of benefits as evaluation measures, but also present
significant challenges in implementation. As a result, states should be thoughtful
about their use and treatment of SLOs in the near-term, recognizing that in the
long-run, with added local capacity, SLOs could more deeply engage teachers in
their evaluation and improve individual and collective teacher capacity.
Benefits:
 SLOs codify a process already embedded in effective teaching – using rich student data to
set standards-based goals and monitoring progress towards those goals. This process
represents a significant and meaningful shift in using high quality practice to inform and
drive policy.

SLOs provide a way of measuring student growth/achievement that can be applied across
a broad spectrum of subjects and grades, without the creation of new statewide
assessments, which is both costly and time-consuming.

SLOs provide an opportunity to take a more robust approach in assessing teacher impact
on student achievement, allowing for broad and specific goals reflecting the unique
context of a school, classroom, and/or teacher.

SLOs allow teachers to take ownership of a portion of their evaluation, which can create
buy-in and add authenticity to the process.
6
Benefits and Challenges, continued
Benefits, continued:
 Use of SLOs can drive professional learning for educators and provide a powerful link
between evaluation and professional development to create a continuous improvement
cycle.

SLOs can directly link the evaluation to the Common Core standards, even outside of
reading and math, by targeting key teaching practices required for effective instruction of
new, more rigorous curriculum
Challenges:

Based on the experiences of early adopter states, most teachers and principals do not
readily possess the necessary data and assessment literacy to develop strong assessments
or SLOs. States must invest significant time and resources to develop educators'
readiness in these areas.

In most states and districts, there is a lack of strong assessment coverage in many grades
and subjects, but also a demonstrated lack of quality in many teacher-created
assessments. States must provide clear training and guidance about the selection of
quality assessments for use in setting SLOs.
SLOs can lower expectations for students if they are not set at a rigorous level. States will
need to provide clear training and guidance as well as mechanisms for monitoring to
ensure SLOs are set at an appropriate and consistent level of rigor for all teachers.

7
Early Adopter Districts
Along with several other large school districts, District of
Columbia Public Schools (DCPS) was an early adopter of SLO
processes as part of its evaluation system. This district provides
important lessons learned and insights to states as they develop
systems for SLOs.
DCPS: The district is in its fourth year of using an SLO process as part of
teacher evaluation. Evaluation outcomes are also linked to compensation.
Teachers in non-tested grades and subjects develop SLOs that count as 10% of
their overall evaluation rating. The district provides a list of suggested
assessments and targets for most grades and subjects, and all assessments
must be rigorous, aligned to standards, and appropriate for the teacher's
students. Targets can be based on either growth or mastery and are
developed collaboratively by the teacher and the principal. When SLOs are
set, a clearly defined rubric is developed to determine a teacher's SLO score
based on student performance.
8
Emerging State Models
Several leading states (DE, GA, IN, NY, RI) have recently begun
piloting the use of SLOs as part of statewide systems of educator
evaluation. These states provide an initial framework for
designing systems of SLOs and offer early lessons learned as they
continue to refine and improve SLO implementation. This
presentation takes a closer look at two of those models.
Indiana: SLOs are used for all teachers in the state default model, while
districts developing their own model can choose to use them. In the default
model, SLOs comprise 10%, 15%, or 20% of the overall evaluation, depending
on the availability of growth model data. State guidance outlines criteria for
assessment selection and specifies an order of preference that requires
teachers use state assessments if available. Teachers must select a learning
objective that applies to all students in a class and a targeted objective focused
on low-performing students. SLOs are set collaboratively among teachers and
approved by principals.
9
Emerging State Models, continued
Rhode Island: All teachers must set between two and four SLOs. For
teachers in non-tested grades and subjects, SLOs will comprise 51% of
their evaluation. SLOs will be combined with value-added data for
teachers in tested areas. Teacher and school-developed assessments must
be reviewed on a quality rubric. Student learning targets must be aligned
with school-level goals and be horizontally and vertically consistent across
a school. Targets should be rigorous but attainable and if met, indicate
that students have the essential knowledge and skills for the next level of
instruction. Teachers and evaluators agree on SLOs at a beginning-of-year
conference and meet mid-year to discuss progress and determine if any
adjustments are needed.
10
Student Learning Objectives Framework
Designing a student learning objective process requires
states to address four core areas, as well as build capacity
and continuously improve systems over time:
I. Selecting assessments
II. Setting targets
III. Reviewing results and scoring
IV. Providing state supports
In approaching this work, states must consider their objectives for
the SLO process, what they will measure, how SLOs will be
incorporated as part of an evaluation and growth system, and how
to maximize the quality of the system over time.
11
Student Learning Objectives Framework
Selecting Assessments
•Relevance
•Stretch
•Precision
•Rigor
•Attribution
•Basis for Measurement
•Timeliness
•Adaptability
Setting Targets
•Basis for Measurement
•Type
•Instructional Interval
•Student Population
•Breadth
•Relevance
•Consistency
•Comparability
•Timeliness
•Rigor
Reviewing Results and Scoring
•Roles and Responsibilities:
Evidence and Teacher and
Leader Input
•Scoring Process: Assessment of
Progress, Weighting, and
Overall Scoring
•Supporting Continuous
Improvement: Scoring Security,
External Validation, and
Ongoing Development
Providing Systems and Supports
•Analyzing Readiness
•Capacity Building
•Providing Strategic
Communication
•Monitoring
•Ensuring Continuous
Improvement
•Soliciting Stakeholder Feedback
12
I. Selecting Assessments
The first step in measuring student learning is to select appropriate
assessments or sources of evidence. Assessments may vary in format
and design, but should meet minimum standards. Further, states must
recognize that intensive professional development to support
assessment literacy is essential in supporting this process.
Key criteria include:
 Relevance: Assessments should reflect knowledge and skills that are valuable to the
student and school context, including alignment to state standards, the Common Core,
and/or national standards for certain subjects (e.g. career and technical education).
Additionally, assessments should align with goals for the student, school, or district.
 Stretch: Assessments should measure student performance across a wide range of
performance levels to ensure that they provide an accurate measure of student
achievement/growth, including for those who perform significantly above or below
grade level.
 Precision: Assessments should be valid, accurate, and yield specific scores. While the
psychometric standard of validity is not necessarily required, there should be evidence
to suggest that the assessment is strongly aligned with course content and have
enough variation in scale to yield scores that are meaningfully differentiated across
performance levels.
13
I. Selecting Assessments, continued
Key criteria, continued:
 Relevance: Assessments should reflect knowledge and skills that are valuable to the
student and school context, including alignment to state standards and the Common
Core.
 Rigor: Assessments should capture true mastery of skills aligned to college and careerready expectations and higher order thinking skills, such as critical thinking, problem
solving, collaboration, communication, etc.
 Attribution: Assessments should yield data that can be assigned to individual teacher
efforts (i.e. assessments must yield scores at the individual student level that can be
linked to individual teachers based on state/district business rules).
 Basis for Measurement: Assessments used to measure student growth should include
a way to develop baseline data to benchmark progress (i.e. there should be some sort
of pre-test and post-test available if a student learning objective is measuring growth).
 Timeliness: Assessments should provide data in time for use in evaluations. (Some
assessments such as AP tests that return results in July may not be able to be used for
student learning objectives depending on the evaluation cycle and system design of
the state/district).
 Adaptability: Assessments should include the potential for accommodations and
modifications for students with disabilities and English language learners.
14
I. Selecting Assessments, continued
Key issues to consider:
 Determine what types of assessments qualify, for example:
 common assessments aligned to state standards,
 assessments purchased or created by schools or districts,
 classroom assessments, which may include teacher-created assessments,
 project-based assessments,
 performance-based assessments.
 Establish and clearly articulate an order of preference for use of different
assessment types.
 Consider how the state will address the issue of assessment cost.
 Consider how teachers, schools, and districts can work collaboratively to
develop assessments.
 Determine a process for approval of assessment selection. (Will the state
play a supportive role by providing guidance and training to districts, or
will it exercise stronger authority?)
15
DCPS Assessment Selection
DCPS provides a list of suggested assessments for most grades and
subjects, and outlines key criteria for assessment selection, including:
• Alignment: Must be aligned to state standards and appropriate for
teacher's class and students; exemplar/recommended SLOs developed by
district for most subjects/grades.
• Stretch: Not addressed, though district provides guidance on setting
appropriate targets for students at various performance levels.
• Precision: Not addressed.
• Rigor: Must include multiple question types including higher levels of
Bloom's taxonomy, real world application or problem-solving, and brief
constructed response.
• Attribution: Required.
• Basis for Measurement: Performance tasks must have a pre-test.
component.
• Timely: Required.
• Adaptable: Required.
16
Indiana Assessment Selection
State guidance for the default model specifies that teachers use state
assessments if available, or, if not, common school or district assessments,
or individual classroom assessments, if there are no other teachers in the
content area. The state provides a pre-approved list of assessments, and
requires evaluators approve any additional assessments. State guidance
outlines key criteria for assessment selection:
•
•
•
•
•
•
•
•
Alignment: Must cover most key content standards; the state requires
completion of the Alignment and Coverage Check Chart.
Stretch: Must assess pre-requisite objectives from prior years and objectives
from the next year's course.
Precision: Must adhere to state established order of preference.
Rigor: Required – the state provides an Assessment Rigor Analysis chart.
Attribution: Required.
Basis for Measurement: Required in target-setting guidance.
Timely: Must provide score reporting in time for evaluation.
Adaptable: State provides guidance for local decision-making.
17
Rhode Island Assessment Selection
Rhode Island requires all teachers to set between two and four SLOs.
Teacher and school-developed assessments must be reviewed on a quality
rubric reflecting the following criteria:
• Alignment: Must be aligned to state standards (and show evidence of this
alignment); must be paced with curriculum; common assessments must be
used if available.
• Stretch: Must include varied response types (e.g. multiple choice, constructed
response, etc.) to assess complexity of content.
• Precision: To ensure validity, must include at least three versions of a test if
given multiple times throughout year.
• Rigor: Must include some evidence of higher-order thinking.
• Attribution: Required.
• Basis for Measurement: Required in target-setting guidance.
• Timely: Must provide score reporting in time for evaluation.
• Adaptable: Must make accommodations available to ensure equitable
testing.
18
II. Setting Learning Objective Targets
Once a high quality assessment has been selected, it is essential to develop
strong student learning objectives that set ambitious but achievable
expectations for students and that provide teachers with clear instructional
goals. SLO processes should clearly articulate a process for setting targets
that identifies who is responsible for setting and approving targets and
what steps are in place to ensure quality. States should also ensure training
and professional development
Key considerations include:




Basis for Measurement: Targets should distinguish different levels of performance
mapped to a clear rubric (i.e. targets must set specific student performance
expectations with clear thresholds).
Type: States/districts must determine if targets can be based on student
achievement (based on mastery of course content) and/or student growth
(improvement in performance using baseline data).
Instructional Interval: Targets should specify the duration of instruction during
which performance will be measured.
Student Population: States/districts should determine if targets should be set for
all students, particular subgroups of students, or both.
19
II. Setting Learning Objective Targets, continued
Key considerations, continued:






Breadth: States/districts should determine if targets should address all content
standards for the course, or if a subset is permissible. (How much of course
content standards must be covered to be sufficiently meaningful while ensuring
targets are not overly broad/vague?)
Relevance: Targets should be meaningful to school and student context.
Rigor: Targets must be set at a level that corresponds with ambitious but
achievable outcomes.
Consistency: States/districts/schools may want to require or recommend that
teachers of the same grade or subject have the same or similar targets. (Vertical
or horizontal alignment of targets can increase quality and provide greater
cohesion.)
Comparability: States/districts should consider strategies to make student
learning targets comparable to other measures of student growth.
Timeliness: Targets must be set by a particular point in the school year and
measured at a particular point(s) in the year (i.e. approval must occur within a
certain timeframe and final evaluations must reflect on data outcomes by a
certain deadline).
20
II. Setting Learning Objective Targets, continued
A clear process for setting and approving SLOs must be
developed that articulates requirements and areas of flexibility.
 What is the process for setting objective targets?
 Who sets them?
 Who approves them, and what's the process for ensuring quality?
 How are teachers involved?
 How clearly articulated are different levels of performance? (Will the state
provide standards for each level? Exemplars?)
 What is the process for tracking teacher progress toward targets and
adjusting instruction as necessary?
 Is there a process for revising learning objectives mid-year if it becomes
clear they can be improved? In what circumstances is this appropriate?
21
District of Columbia Targets
DCPS requires principals to review and approve all SLOs to ensure that
they meet quality criteria and also conducts a central office review of
feasibility. The district provides sample SLOs for most grades/subjects.
 Measurable: Targets can be mastery or growth and must be anchored in
baseline data.
 Breadth: Targets must cover all students that a teacher instructs and cover
the majority of content for a given course.
 Rigor: Targets must represent an ambitious but achievable level of
performance and should set a goal of 1.25 years of academic growth.
 Meaningful: Targets should be based on the specific needs of the students a
teacher instructs.
 Consistency: Sample targets are provided for most subjects and grades. ELL
targets must be based on state AMOs. Standard rubrics are available for
some subjects.
 Timing: Targets must be set at initial conferences (by October) with data
available in time to inform a final evaluation (in June). DCPS reviews each
target for feasibility (not quality).
22
Indiana Targets
Indiana provides teachers and evaluators with a checklist of key steps for
determining targets. It also provides exemplar and non-exemplar SLOs.
 Measurable: Targets must be measurable and based on previous performance.
Targets focus on student mastery.
 Breadth: Teachers must select a learning objective that applies to all students in
a class and a targeted objective focused on low-performing students. State
guidance outlines three levels of student preparedness on which to base
targets. Targets should cover all major standards and skills.
 Rigor: Relies upon the judgment of the teacher and evaluator.
 Meaningful: In setting targeted objectives, teachers must consider those
students that enter the course inadequately prepared and may choose to focus
on a few key standards or all standards.
 Consistency: Teachers in subjects/grades covered by an approved assessment
must adhere to state established content mastery standards. The state provides
guidance on mastery using other assessments.
 Timing: Targets must be set at initial conferences (by October), with a midcourse check-in to discuss progress (in January -February).
23
Rhode Island Targets
Rhode Island provides teachers and principals with a checklist of criteria to
evaluate targets. The state also provides exemplar and non-exemplar SLOs.
 Measurable: Targets must be measurable and based on baseline/historical data
or previous classes' performance. Targets can be mastery or growth.
 Breadth: Targets must cover all students, but can include multiple tiered groups
for subpopulations of students. Targets should cover all major standards and
skills.
 Rigor: Targets must correspond with at least one year of learning and
attainment must suggest that students are on track in that subject/course.
 Meaningful: Targets must include a rationale for the selection of the target
based on student needs.
 Consistency: Teachers across subjects/grades within a school should have the
same targets. Targets should be vertically aligned with school and district
priorities.
 Timing: Targets must be set at initial conferences (by October) and then revised
(in January) if there are major changes in class composition or other factors that
would necessitate a change.
24
III. Reviewing Results and Scoring
Following completion of the SLO process, evaluators and teachers must review
SLO results and score the teacher's performance. This process should be
articulated when targets are set. Depending on the level of local control,
processes may be established by the state or district, and may even be subject
to collective bargaining.
Key considerations include:





Evidence: Articulate the responsibilities of teachers and evaluators in gathering and
evaluating evidence of student performance and what amount of evidence is
sufficient.
Teacher and Leader Input: Determine what level of professional judgment is to be
used in making judgments and what the process is for teacher feedback and input.
Assessment of Progress: Articulate a process for making a judgment about teacher
performance on student learning objectives, including who makes the judgments, by
when, and whether other evidence or information should be considered in making a
scoring determination.
Weighting: If teachers set multiple SLOs, determine how these should be weighted.
Overall Scoring: Determine how judgments and evidence from SLOs are incorporated
into the larger summative evaluation.
25
III. Reviewing Results and Scoring, continued
Key considerations include, continued:




Scoring Security: States/districts may want to establish systems to ensure greater
scoring security.
External Validation: States should consider whether to develop a process to validate
ratings, such as through correlational analysis with other measures or spotchecking/auditing of results at the school or district level.
Ongoing Development: States/districts must articulate how the SLO process is linked
to feedback for professional development opportunities based on results.
Stakeholder Feedback: States/districts should develop systems to gather stakeholder
feedback throughout the SLO design and implementation process to inform
continuous improvement.
26
IV. State Systems and Supports
While the role of the state may vary considerably depending on the context
and degree of local control, there are a set of core functions and supports that
a state should anticipate providing in order to implement a successful SLO
process, including:





Analyzing Readiness: States should be prepared to analyze district readiness in terms
of the level of assessment and data literacy among teachers and principals, and use
this information to shape and differentiate training and professional development.
Capacity Building: States should anticipate providing or facilitating high-quality
training on assessment selection, assessment creation, use of data, and goal-setting.
Providing Strategic Communication: States should develop a strategic
communications plan for educator evaluation efforts of which SLOs are an important
component.
Monitoring: States should develop policies and protocols to monitor and assess the
quality of SLO implementation to ensure SLO outcomes align with other evaluation
data, are not disproportionally impacting certain teachers, and are being
implemented with rigor and fidelity.
Ensuring Continuous Improvement: States should develop a continuous improvement
process to evaluate and refine the SLO process over time. This process should
address perceived deficiencies as well as look for ways to enrich the SLO process.
27
DCPS Reviewing Results and Scoring
DCPS principals judge evidence of student learning through SLO outcomes and
assign a rating aligned with the rubric established at the initial conference.

Evidence: Teachers must provide evidence that student performance has met
targets at their year-end conferences with evaluators.

Assessment of Progress: DCPS provides a general rubric with suggested criteria for
each rating level. Principals and teachers agree on a rubric at the beginning of the
year. At the end of the year, principals validate student scores and assess student
progress against this rubric.

Teacher and Leader Input: Teachers input is given when SLOs are set (at an initial
conference).

External Validation: Not addressed.

Overall Scoring: SLOs are rated according to a 4-level rating scale that is specified at
the initial conference.

Ongoing Development: Principals and teacher discuss student learning outcomes at
the end of the year to identify a final rating and determine professional growth
strategies. This supplements ongoing conversations with principals and master
educators on observational feedback.
28
Indiana Reviewing Results and Scoring
Indiana evaluators judge evidence of student learning through SLO outcomes
and, following discussion with the teacher, assign a rating based on professional
judgment.

Evidence: Teachers must provide evidence of SLOs 48 hours prior to their summative
conference with evaluators. Evidence of class objectives should only include student
scores on the assessment, while targeted objectives may also require additional evidence,
such as class work, student projects, etc.

Assessment of Progress: Evaluators use professional judgment to evaluate student
learning in comparison to established targets and then determine a rating, based on four
performance levels, for each objective.

Teacher and Leader Input: SLOs are set collaboratively among teachers and approved by
evaluators.

External Validation: Not addressed.

Overall Scoring: All teachers receive SLO ratings on a 4-point scale. A weighted average is
used to determine the final SLO score. (Class and targeted objectives count equally.)

Ongoing Development: Evaluators and teacher discuss student attainment of learning
objectives, and the evaluator establishes ways to support the teacher.
29
Rhode Island Reviewing Results and Scoring
Rhode Island principals judge evidence of student learning through SLO
outcomes and assign a rating based on professional judgment.
 Evidence: Teachers must provide evidence of SLOs prior to their final
conferences with principals.
 Assessment of Progress: Evaluators use professional judgment to evaluate
student learning in comparison to established targets and then rate
performance against all targets on a five-point scale.
 Teacher and Leader Input: Teacher input is given when SLOs are set (at an
initial conference).
 External Validation: Not addressed.
 Overall Scoring: All teachers receive SLO ratings on a 5-point scale based on
professional judgment of their performance across 2-4 SLOs.
 Ongoing Development: Evaluators and teacher discuss student attainment of
learning objectives, what strategies were effective and ineffective, and what the
teacher learned about curriculum, assessments, and instruction through the SLO
process.
30
A Key Tension: Standardizations v. Individualization
In designing a process for SLOs, states must balance the desire for
consistency and comparability of rigor with the desire to promote teacher
buy-in, engagement, and professional learning by allowing educators to
select their assessments and targets.
 Georgia designed an SLO process that requires districts to select assessments and
targets for each grade and subject that are then implemented by all teachers across
the district. This ensures that all teachers are setting rigorous targets that are
comparable, but does not allow for significant engagement, use of baseline data, or
personalization.
 DCPS allows teachers to set their own targets and select assessments (subject to
principal approval). While DCPS works to boost comparability and rigor by providing
extensive training and SLO models for each grade and subject, there is a significant
amount of variability in the system, and the district limits the weight of SLOs to 15% of
the evaluation system.
 Some states, like Delaware, have sought balance on this spectrum by employing SLO
rubrics, pre-approved lists of assessments, and rich guidance and state supports as a
means to ensure a sufficient level of quality and consistency while promoting as much
flexibility and authenticity as possible for teachers.
31
The Potential for SLOs in the Long-Term
Over time, SLOs can evolve through continuous improvement
and thoughtful phase-in to allow for greater teacher engagement
and more specific development of individual and collective
capacity.
 In the early years, while educator assessment and data literacy is limited,
states may want to design SLO processes that are grounded in specific
criteria with a narrower range of options for assessments, targets, and
scoring.
 However, as educators gain expertise in this process and as districts
demonstrate increased capacity, states could consider loosening
requirements to allow for more personalization of the system to promote
greater educator ownership of the evaluation and to more fully incorporate
professional development goals.
32
Questions?
Questions?
Robin Gelinas Berkley, EducationCounsel:
robin.berkley@educaitoncounsel.com
Margie Yeager, EducationCounsel:
margery.yeager@educationcounsel.com
33