Student Growth Measures in Teacher Evaluation
Facilitator’s Guide for Training Module 4:
Scoring an Individual SLO
Overview
Training Purpose and Goals
The student learning objectives (SLO) training is intended to assist teachers and administrators in
implementing SLOs in their schools. At the conclusion of the four training modules, teachers
should be able to

Explain what makes a high-quality SLO.

Select or develop appropriate assessments.

Develop growth targets.

Score an individual SLO.
Facilitators should be able to

Review the components of the Ohio Teacher Evaluation System Framework and Model.

Deconstruct Ohio’s model for developing SLOs.

Understand the tools and processes used in developing, organizing, and reviewing SLOs.

Use strategies to facilitate work on SLOs in your school district or school.
Audience
This facilitator’s guide is designed as a blueprint for the training that will be delivered to teachers
within their schools to prepare them to develop and implement SLOs this year (2012–13),
whether SLOs are being implemented on a pilot basis or a fully operational basis within the
evaluation system. Facilitator notes, as well as suggestions of the direct language to use in
conveying the material to teachers, are provided.
Timing
The training that you will deliver is split into four modules. Each module is designed to be
delivered in 60 minutes, during time designated for professional development or common
planning.
Resources
The following resources are provided for use in delivering the four training modules to staff
within each school:

Facilitator’s guide (this document)

Slide presentation

Participant handouts

Tips and tools for presenting the module, which includes an annotated agenda and
anticipated questions.
Facilitator’s Guide: Module 4—2
Why Student Learning Objectives?
Recent federal and state policies have led to new systems of educator evaluation. These new
systems include multiple measures, one of which is often student learning or student growth.
SLOs have recently come to the forefront of this work and have been highlighted as a promising
approach for measuring student growth, particularly for the majority of teachers in subjects or
grades not covered by a standardized assessment.
The Ohio Context
We know that a well-designed evaluation system can support professional growth and improve
educator practice, which, in turn, leads to better student outcomes. In Ohio, as elsewhere, a major
challenge in creating new evaluation systems is to figure out how to measure student growth for
the majority of teachers and students for whom state-administered standardized tests or relevant
and validated vendor assessments are not available. An additional challenge is incorporating
measures of student growth within the evaluation system in such a way that directly ties this
information to the instructional planning and goals of teachers and represents the unique needs
and goals of the school district.
Advised by the Ohio Department of Education (ODE), many school districts will implement
SLOs as a part of the “locally determined measure” portion of the student growth component,
which is 50 percent of the overall evaluation per legislation. If value-added or approved vendor
assessments are available, those options must be used, and locally determined measures such as
SLOs will account for up to 40 percent of the overall evaluation. In grades or subjects where
value-added or approved vendor assessments are not available, locally determined measures may
account for up to 50 percent of the evaluation, which is the entire student growth component.
Some school districts will pilot SLOs in the 2012–13 school year, whereas other school districts
will fully implement SLOs within their teacher evaluation system.
Facilitator’s Guide: Module 4—3
Slide 1
Slide 1 is the title slide for Module 4.
Slide 2
Review the training norms with the group.
Slide 3
Plan to spend 15-20 minutes on slides 1–11.
Say:
“Now we are proceeding to the fourth and final module of the training on
student growth measures and student learning objectives.”
Slide 4
Say:
“The previous three modules have covered the first four steps in the SLO
development process. This module will focus on the fifth step, which is to
monitor and prepare for the SLO rating and discussion, or what we are
referring to as “scoring.”
“In this final stage, the SLO has been set, reviewed, and approved, and now
you are gathering and analyzing evidence of progress toward the goal, and
organizing and presenting evidence of final attainment of the goal.”
Facilitator’s Guide: Module 4—4
Slide 5
Say:
“First, let’s review the timeline in which SLO scoring will take place. For
2012–13, the writing, approval, and revision time is shortened, so that SLO
submission is occurring by December 14; reviews and revisions, if
necessary, are occurring in December; and the SLO is final by January. In
future years, the SLO should be set and final somewhat sooner, in
November. Both this year and moving forward, the SLO calculation of
attainment and scoring must be completed prior to the May 1 deadline. So,
as we are discussing the processes of monitoring progress, gathering and
presenting evidence, and scoring SLOs, this is the date to keep in mind from
which we are working backwards. Also, note that ample time for committee
or evaluator review, per your district’s requirements, must be built in prior
to this date to ensure that the entire teacher evaluation process is complete
by May 1.”
Slide 6
Say:
“In a moment, we will go through the steps leading up to SLO scoring, but
so that we can all visualize what we’re heading toward, let’s define what we
mean by an SLO score.”
“Simply put, the SLO score is the percentage of students that met their
growth target. The percentage then falls into one of five categories: Most
Effective (5), Above Average (4), Average (3), Approaching Average (2),
and Least Effective (1).”
“These numerical ratings will then be combined with other student growth
measures in the system to arrive at a final rating. We will review this
process at the end of this hour, but first, let’s go through a few simple
examples.”
Facilitator’s Guide: Module 4—5
Slide 7
Say:
“Consider, for example, a teacher who has 100 students in his or her SLO.
If 90 of those students meet the growth target and 10 don’t quite reach it,
this means 90 percent met the target, which is the threshold for the rating of
Most Effective, or 5.”
Slide 8
Say:
“Consider an example where of 89 students in an SLO, 74 meet the growth
target. Seventy-four out of 89 is 83 percent, which corresponds to a rating
of Above Average. As you can see, each calculation is simply the
percentage of students meeting the growth target.”
“In this example of 89 students, below are the thresholds for each
performance level. As you can see, 27 of the 89 students would have to
miss the growth target in order for the rating to be below the “Average”
threshold. So, while it is important to set accurate growth targets, there is
room in this process for some error and adjustment as everyone learns to
gauge accurate growth in their content and with their assessments.”
“Now, let’s outline the process that you will go through to arrive at one of
these five ratings for an SLO.”
Slide 9
Say:
“Backing up for a moment, after the SLO has been reviewed and approved,
the final assessment that will be used to demonstrate growth and the
standards and content that you are going after will be set. After review and
any needed revisions, the teacher should walk away from the approval with
an SLO checklist that has every applicable box checked and every criterion
in the seven components satisfied. It is then up to the teacher or teacher
team to decide the best ways to monitor progress toward that goal and
attainment of those standards. Some type of formative or interim
assessments should be used to monitor student progress. A midcourse
check-in with an evaluator or appropriate committee, [depending on your
district circumstance] is recommended so that the teacher has an
opportunity to discuss student progress.”
Facilitator’s Guide: Module 4—6
“The teacher should bring any information to the check-in that will help the
evaluator gauge progress toward the learning target. An example would be
any student work products or outcomes of the formative or interim
assessments that can show development of the skills or content knowledge
necessary to reach the final goal.”
“If students seem to be off track, the teacher should identify those specific
students and come prepared to discuss ways to change instruction to
facilitate making up ground.”
Slide 10
Say:
“Now we’ll move into discussing how to prepare for the final scoring
process. Again, this will occur at the end of the interval of instruction. In
general, something to remember is that much of the onus of preparing the
evidence of whether students met their targets or not does falls on the
teacher. To make this review process possible, it is helpful to follow these
steps exactly so that every time an evaluator or local committee is looking
at this evidence from one teacher to another, they know exactly what they
are looking at because it is organized in the same way. The more
streamlined the process, the more likely it is both efficient and fair.”
Slide 11
Say:
“There are several documents that the teacher should have ready in order to
present the evidence for review. The evidence should include:

A copy of the original, approved SLO template and checklist for
reference.

The completed Individual SLO Scoring Template, which we’ll review
in a moment.

Copies of the student assessments on which attainment of the growth
targets was measured (not completed student versions, just the
assessments themselves),

An explanation of how the assessment was scored. For example, if
multiple assessments were used, detail how the scores of the
assessments were combined to arrive at the final score. If any rubrics
were used, provide copies. If team scoring was used, describe that
process.”
Facilitator’s Guide: Module 4—7
Slide 12
Plan to spend 15-20 minutes on slides 12–16.
Say:
“After the final assessment has been administered and results are available,
the teacher will complete the Individual SLO Scoring Template to present
the evidence on growth target attainment for each student. The Ohio
Department of Education (ODE) is in the process of making a final version
of this spreadsheet, but the one we are showing you here gives you a sense
of where they are headed in their thinking. The scoring template is
formatted in an easy-to-use spreadsheet. A screenshot of the scoring
template is provided on Handout 4.1, along with the SLO Scoring
Matrix. This template will help you do the simple calculations involved in
arriving at the final score, but its main purpose, really, is to present this
information in a standardized way from one teacher to the next.”
Slide 13
Say:
“Handout 4.2 provides instructions for completing the Individual SLO
Scoring Template in eight steps.”
This slide details the steps in completing the Individual SLO Scoring
Template.
Review each step, referring to the slide and pointing to the applicable
columns on the scoring template as you speak, if possible.
Say:
“Once you have completed the Individual SLO Scoring Template, it is
useful for presentation of the data to sort the scoring template by growth
target tiers, then by students who have met or not met the target. This
information will be useful during the conversation about how the SLO
results can be used to think about changes in practice and instructional
strategies that might be used in the future to reach any identified groups of
students who did not meet the growth targets.”
Ask the group if there are questions on the process, including what data
should be entered into each cell.
Facilitator’s Guide: Module 4—8
Slide 14
Say:
“Clearly, as with any assessment, the more students included in the SLO,
the less any one student contributes to the teacher’s overall rating. With any
SLO, multiple measures of student performance or assessments are helpful
to increase the reliability of the overall measure, but they can become
especially important as fewer students are included in an SLO. For example,
you would want to avoid setting an SLO that included 10 students and only
one multiple-choice assessment taken at one point in time. In that scenario,
much of the final SLO score would hinge on each student’s performance at
that one point in time. There may be situations where it is appropriate for a
teacher to only have 10 students in an SLO because he or she is responsible
for facilitating the learning of those 10 students in that context. In that case,
the evaluator or local committee would want to ensure that the SLO was
capturing multiple measures of each student’s learning.”
“Consider a scenario where 10 students are all in two different courses—
one course with 120 students and one course with 40 students. Suppose that
the assessments in each of the courses are centered around writing skills,
and these 10 students have writing deficiencies. Suppose both teachers did
not set realistic growth targets for these students. While it is important to
carefully set growth targets for every student, the teacher with 40 students is
more affected by the SLO rating by setting those 10 inaccurate growth
targets. So, in cases where the number of students covered by an SLO is
comparatively low, multiple measures and special attention paid to the
accuracy of growth targets are important.”
Slide 15
“Before we move on, are there any questions? Now we’re going to discuss
an SLO scoring example. As we read this example, consider, with the
information you’re given, what aspects of the SLO appear to be strong.
Also consider any improvements that might be suggested to Mr. Ramirez
when setting the SLO growth targets next time.”
Note: If enough time in the session remains, the facilitator can choose to do
this activity as a “fish bowl,” and have three participants come to the front
of the room and have their conversation around this example. One of the
participants should record the key thoughts or themes that come up as these
two questions are answered.
Otherwise, read the SLO example aloud to the group, and facilitate a
discussion around the provided questions.
Example 1 Background: Expect groups to indicate that all components of
the SLO seem to be high-quality multiple measures within the assessment,
Facilitator’s Guide: Module 4—9
and a large majority of students are meeting their growth target. Because
the six students who did not meet their growth target are in bottom of the
middle tier, participants may note that in future years, the teacher could
further refine the growth targets by potentially setting a fourth growth
target tier or providing targeted assistance to students in this section of the
achievement spectrum. The teacher may want to examine the assessments of
these students to determine if any trends in content misunderstanding or
skill deficiencies exist within the group.
Slide 16
Say:
“Now, in groups of three or four, read through each of the remaining
examples describing SLO scoring outcomes. Discuss your answers to the
provided questions. Designate one person within your group to record your
responses and another person to share during our large group discussion.
We’ll reconvene the large group in about 15 minutes.”
Example 2 Background: Expect groups to voice that this SLO was based
on one assessment that was administered without direct pretest data to
gauge growth. Expect participants to talk about increasing the ways in
which calculus students are assessed in order to gauge growth in multiple
ways or find more reliable ways to gauge growth by incorporating multiple
sources of pretest data. Participants might also mention monitoring student
growth more closely during the course to track progress.
Example 3 Background: Expect groups to voice that in subsequent years,
the assessments for these two SLOs would want to continue to be vetted to
make sure they are rigorous and are including grading practices that
reduce subjectivity. Also, expect participants to note that with smaller
numbers of students, even one student missing the growth target can quickly
reduce the overall percentage attaining the goal. This is an issue to be
aware of in terms of stability of the measure, hence making the multiple
measures important.

Examples continue in the next column.

Spend approximately 15 minutes on the large group share-out if
rich discussion is occurring. When the discussion concludes and all
concerns have been addressed, move on to the final section on
student growth measure scoring.
Example 4 Background: Expect the
groups to voice concern that of those
not meeting their growth targets, a
majority were higher achieving
students from the top tier of expected
growth. High-achieving students may
be already meeting a level of
proficiency that is acceptable, but are
not able to stretch to meet targets for
growth as easily as students in the
middle of the achievement spectrum.
With this particular teacher, the issue
may be that (1) the growth targets
were too ambitious or (2) the students
were not receiving enough
enrichment to propel them past a
certain level of proficiency. This issue
could be dealt with in subsequent
years by either setting growth targets
more conservatively on the high end
of achievement or ensuring that
instruction throughout the course is
assisting already high-achieving
students.
Facilitator’s Guide: Module 4—10
Slide 17
Plan to spend 15 minutes on slides 17–21.
Say:
“Now we’ll move on to the final section, which describes the process
through which SLO scores will be combined with other student growth
measures, if applicable, and entered into the online system.”
“In order to provide a consistent process for student growth measure scoring
and minimize the burden on local education agencies (LEAs), each LEA
will enter teacher scores within the Ohio Teacher and Principal Evaluation
System (eTPES), which will then automatically record and calculate overall
scores. The district plan will already be submitted in eTPES, which will
have provided the default percentages for how much of the final score will
be attributed to different types of student growth measures based on the
three categories of teachers: A, B, and C.”
Slide 18
Say:
“Now, we return to which student growth measures each category of teacher
has available to them. The following chart is an example that shows a
district that has decided that, when they are available, value-added scores
will account for 40 percent of a teacher’s overall score, while LEA
measures, specifically SLOs, will account for 10 percent. For category B
teachers, or those without value-added but with vendor assessments, those
vendor assessments will account for 30%, and SLOs and Shared Attribution
will account for 10% each. Then for category C teachers, SLOs will count
for 40%, and shared attribution 10%. Note that this is just an example of
how one district could choose to develop their plan.”
“ODE will have entered a teacher roster as well as value-added reporting
into eTPES.”
Facilitator’s Guide: Module 4—11
Slide 19
Say:
“In some cases, districts may want to make changes to the percentage
attributed to each student growth measure per teacher. Ideally, any
variations would be made at the beginning of the year when the district plan
is submitted in the system.
For example, it may be the case that a teacher only has one year of valueadded data, and the evaluator or committee feels it would be appropriate to
shift more of a percentage of the evaluation onto LEA measures. In this
scenario, consistency in variations should be maintained based on
circumstance.”
“For example, if teachers with one vendor assessment are allocated 20
percent to the measure, but teachers with two or more vendor assessments
are allocated 30 percent to the measures, this allocation should be
maintained for all teachers in that circumstance.”
“Also, a percentage shouldn’t be changed for one teacher without
considering if it creates incomparability between the measures used in other
grades and subjects.”
Slide 20
Entering Student Growth Measure Scores
Per Teacher
Say:
• A worksheet similar to the following will be
completed for each teacher. See Handout 4.4.
“This worksheet represents the scores that will be entered for each teacher.
Let’s look at Handout 4.4 for an easier-to-read version of this worksheet.
Value-added data and percentages allocated to each measure from the
district plan will already be populated in the system.
LEA Measure
Value-Added
Score
MRM Composite
_______
URM Composite
_______
Vendor Assessment
Vendor 1
Score
_______
SLO/Other
SLO 1
Score
Shared Attribution
Score
_______
Shared Attribution
_______
Vendor 2
_______
SLO 2
_______
Vendor 3
_______
SLO 3
_______
Vendor 4
_______
SLO 4
_______
Vendor 5
_______
SLO 5
_______
Value-Added
Score
Vendor Assessment
Score
SLO/Other
Score
Shared Attribution
Score
Value-Added
Percentage
Vendor Assessment
Percentage
SLO/Other
Percentage
Shared Attribution
Percentage
STUDENT GROWTH MEASURES =
The greyed-out boxes - the “value added” score and percentage, the “vendor
assessment” score and percentage, the “SLO/Other score and percentage”
and the “shared attribution” score and percentage - will be calculated and
populated by the system, so only the individual scores, 1-5 in each yellowhighlighted section, will be entered by the teacher or LEA representative.
Note that each teacher can incorporate up to five vendor assessments and
six LEA measures (five SLOs and a shared attribution measure) if those
measures are available. So, a teacher who did not have value-added or
vendor assessments available could submit up to five SLOs in the system.
However, all measures within each of these three categories would be
weighted equally in the system. For example, all five SLOs would receive
the same weight.”
Facilitator’s Guide: Module 4—12
20
Slide 21
Say:
“The final student growth measure score will be calculated by the eTPES
system. The calculation used to arrive at this score is still being finalized
and will be approved by the student growth measures advisory committee
soon. The final SGM score will be converted into a rating of Above,
Expected, or Below. We know that this still leaves some ambiguity about
how the final SGM score will be calculated, but there will be additional
materials available online that detail this process once it is approved.”
Slide 22
Plan to spend 5 minutes on slides 22–23.
This slide is a reflection, asking participants to anticipate the stumbling
blocks or concerns that LEAs and teachers will have around the SLO
scoring process and combined student growth measure scoring. Be sure to
ask for ideas about how this training can better address those foreseen
issues or concerns.
Slide 23

Say:

“This concludes Module 4. Thank you for participating.”
Facilitator’s Guide: Module 4—13