DDMs for School Counselors
RTTT Final Summit
April 7, 2014
Craig Waterman & Kate Ducharme
Agenda
Overview of DDMs (30 minutes)
 Introduction to DDMs
 Q&A with Craig Waterman
DDM Considerations for School Counselors –
Group Planning Activity (40 minutes)
Selecting Measures (15 minutes)
Closing and Next Steps
2
Massachusetts Department of Elementary and Secondary Education
The Educator Evaluation
Framework
Everyone earns two ratings
Summative
Performance
Rating
Student
Impact
Rating
Exemplary
Proficient
Needs Improvement
Unsatisfactory
High
Moderate
Low
3
Summative Rating
Two Ratings –
Intersection of Practice and Impact
Exemplary
Proficient
1-yr Self-Directed
Growth Plan
2-yr Self-Directed Growth Plan
Needs
Improvement
Directed Growth Plan
Unsatisfactory
Improvement Plan
Low
Moderate
High
Rating of Impact on Student Learning
Identifying DDMs –
Key Questions
 Is the measure aligned to content?
 Does it assess what the educators intend to teach and what’s
most important for students to learn?
 Is the measure informative?
 Do the results tell educators whether students are making the
desired progress, falling short, or excelling?
 Do the results provide valuable information to schools and
districts about their educators?
See Technical Guide B for more
about these key questions:
http://www.doe.mass.edu/edeval
/ddm/TechnicalGuideB.pdf
Key Messages for Stakeholders
Difference Between Direct and
Indirect Measures
Direct Measure
Direct Measures
Classroombased
Educator
Responsibilities
Classroom-based
educator provides
direct instruction to
students.
Measures of
Student
Learning
Students learn;
acquire knowledge
and skills
 Classroom-based educators can directly measure the
impact of their instruction on student learning.
Difference Between Direct and
Indirect Measures
Indirect Measure
Indirect Measures
SISP
Responsibilities
SISP
educator
provides
specialized
support
services to
students.
Indirect
Measures
Students
can access
general
education
curriculum.
Measures of
Student
Learning
Students
learn;
acquire
knowledge
and skills
 For SISP who do not directly instruct students, there often is an
an intermediary step that exists between SISP responsibilities
and student learning, therefore making the SISP contribution
indirectly tied to student learning.
Do DDMs have to be identical?
 DDMs must be “comparable across schools, grades,
and subject matter district-wide.” (603 CMR 35.09(2)a)
Example
Elementary
Another
Elementary
5th Grade
Teacher
5th Grade
Teacher
Music
Teacher
Music
Teacher
See Investigating Fairness
Implementation Brief:
http://www.doe.mass.edu/edeval/dd
m/Fairness.pdf
Type 1: Comparable within a grade,
subject, or course across schools within
a district
Identical measures are recommended
Investigate Fairness for all students and
educators
Type 2: Comparable across grade or
subject level district-wide
Impact Ratings should have a consistent
meaning across educators; therefore, DDMs
should not have significantly different levels of
rigor
Identify Commonalities
One of the goals of DDMs is to support
common measures across a district
Approach to selecting DDMs for SISP
educators could involve identifying
commonalities across multiple roles
For example, a district-wide DDM involving
collecting feedback from stakeholders on the
quality, usefulness, and timeliness of
communications
10
Massachusetts Department of Elementary and Secondary Education
DDMs for District “Singletons”
 Districts should consider the following options to
identify and develop DDMs for “singletons”:
 Work with neighboring districts to identify and/or create
DDMs.
 Group educators within the district who have similar
responsibilities, albeit different roles, to collaborate on
DDMs.
 Look to outside rating entities (such as state or national
organization standards).
 Use DDMs that can be easily interpreted (e.g., for a
school counselor, using number of seniors who applied
to college instead of number of students who are
prepared to apply to college.)
Massachusetts Department of Elementary and Secondary Education
11
DDM Implementation Plan –
June 1, 2014
Resources:
http://www.doe.mass.edu/edeval/ddm/
 Implementation Briefs
 Technical Guide B
 Webinar Series
 Commissioner’s Memoranda
 Educator Evaluation Newsletter
 Technical Assistance and Networking Sessions
 Using Current Assessments in DDMs (Curriculum
Summit)
 Example Assessments
 Other ESE documents (Technical Guide A, Part VII,
Regulations)
13
Questions and Answers
Who scores the DDM?
 Outside Organizations
See Scoring and
Parameter Setting
Implementation Brief:
http://www.doe.mass.e
du/edeval/ddm/Scoring
&PSetting.pdf
 Commercial assessments
 Automated methods
 Paid raters (e.g., college students, retired
teachers)
Internal
External
 Teams of Teachers (e.g., all 5th grade teachers)
 Team members rate each other’s students’
responses
 Multiple raters score each response
 Individual Teachers
 Random auditing (rechecking)
How do I determine high,
moderate, or low growth?
Qualitative
Ask educators
how much
growth is
moderate on this
assessment?
See Scoring and
Parameter Setting
Implementation Brief:
http://www.doe.mass.e
du/edeval/ddm/Scoring
&PSetting.pdf
Quantitative
Fairness
Identification
Agreement
Variability
Educator
Review
Previous
Results
Does the DDM
make
meaningful
distinctions? Do
all students have
an equal chance
to demonstrate
growth?
DDMs for School
Counselors
Identifying DDMs –
Key Questions for Specialized Instructional Support
Personnel
 Is the measure aligned to content?
For SISP educators (including school counselors),
content should reflect their job functions and
responsibilities and should align to what they do to
support students, educators, administration, and/or
parents.
 Instructions: With you table, identify key content
(i.e., key job functions and responsibilities) of your
role.
Massachusetts Department of Elementary and Secondary Education
18
Identifying DDMs –
Key Questions for Specialized Instructional Support
Personnel
What does success look like?
What would you see if someone was successful at
these job functions and responsibilities?
 Instructions: With your table, identify what success
looks like for each of the job functions and
responsibilities you identified.
19
Massachusetts Department of Elementary and Secondary Education
Identifying DDMs –
Key Questions for Specialized
Instructional Support Personnel
 What is measureable?
 Measurement is…
The systematic process to assign a number to an
observation
 Example: How tall is my friend?
1.
2.
3.
Place a measuring tape next to my friend and read the number next to the
line that is closest to the top of his or her head.
Stand next to my friend and estimate how much taller or shorter they are
compared to myself.
Place a ruler flat on my friend’s head and mark a line on a wall where the
ruler hits the wall. Then measure how far the line is off the ground.
 Instructions: With your table, identify approaches
could you use to measure the job responsibilities
and functions you identified.
Identifying DDMs –
Key Questions for Specialized Instructional Support
Personnel
Is the measure informative?
Chosen measures should provide both
educators and districts with actionable
information to inform practice and identify
both areas of strength and areas where
more supports are needed.
21
Massachusetts Department of Elementary and Secondary Education
Key Messages for Stakeholders