District-Determined Measure Example
Social-Emotional Skill Development in the Therapeutic Setting
Content Area and Grade Range: School Psychologists, grades 6-12
DDM Summary: This DDM measures growth in grade 6-12 students’ ability to identify
their current emotions and corresponding physical symptoms, and to identify and apply
a healthy coping strategy to improve their immediate emotional state in the therapeutic
setting.
Developed by: Sarah Auger, School Psychologist (Stoneham Public Schools, MA) and
Carly Greenstein, School Psychologist (North Reading Public Schools, MA)
Reviewed by: Names and roles of the individuals who reviewed the measure
Pilot Districts: North Reading Public Schools, Stoneham Public Schools
Date last updated: June 2015
Table of Contents
Introduction ................................................................................................................... 2
Instrument...................................................................................................................... 4
Administration Protocol ............................................................................................... 4
Scoring Guide .............................................................................................................. 11
Measuring Growth and Setting Parameters .............................................................. 13
Piloting ......................................................................................................................... 13
Assessment Blueprint ................................................................................................ 16
SEEM Collaborative – DDM – School Psychologists 6-12 1
Introduction
Description of the Measure
This DDM is a measure of the school psychologist’s direct impact on students’ growth in
their ability to identify their current emotions and corresponding physical symptoms and
to identify and apply a healthy coping strategy to improve their immediate emotional
state in the therapeutic setting. It is a core function of the school psychologist to support
the development of students’ social/emotional skills in these areas, particularly because
students who struggle with these skills often experience challenges in school and are
subsequently referred to the school psychologist for intervention support. This DDM
may also be used by other personnel who instruct and support these students, such as
school guidance or adjustment counselors.
This measure is designed for school psychologists who work regularly and intensively
with a small caseload of students who receive frequent, ongoing social/emotional
intervention and support. When assessing the progress of a small sample of students, it
is important to collect sufficient evidence to make valid inferences about these students’
social/emotional growth over time. As a result, this DDM includes multiple measures
that are administered throughout the school year. The first is administered to students
as a pre- and post-assessment at the start and end of the school year, while the other
two solicit students’ journal reflections and the psychologist’s student observations and
are administered on a monthly basis. These measures assess only a portion of the
content taught by the school psychologist; however, they were chosen as the focus for
this DDM because they require students to identify, describe, and analyze emotions,
experiences, and coping strategies to improve their immediate emotional state within
the therapeutic setting – key skills at the center of the school psychologist’s instruction
and support.
Potential Modifications
School psychologists with larger caseloads may opt to use this DDM with a randomly
selected small sample of three to five students. Alternatively, if they measure growth
with their full caseload of students, they may be able to draw valid inferences with fewer
data points so may decide to use only two of the three measures described here or,
alternatively, to collect data quarterly or every two months instead of monthly. If School
Psychologists are unable to administer all three measures, he/she may choose to use
the journal prompts as a singular measure to assess students’ growth. This would
impact scoring and overall growth parameters.
DDM Objective
This measure is aligned to the following Core Course Objective (CCO): Students will
identify their current emotions and corresponding physical symptoms, then identify and
apply a healthy coping strategy to improve their immediate emotional state in the
therapeutic setting.
SEEM Collaborative – DDM – School Psychologists 6-12 2
Content (Standard)
Weight
CCO: Students will identify current emotions.
Mental Health Standard 5.1- By the end of grade 5, students identify the
various feelings that most people experience and describe the physical and
emotional reactions of the body to intense positive and negative feelings
Mental Health Standard 5.7- By the end of grade 8, students will identify and
describe the experience of different feelings (such as elation, joy, grief, and
rage) and how feelings affect daily functioning (MA Comprehensive Health
Curriculum Framework)
25% of the
measure
CCO: Student will link current feeling to corresponding physical symptoms.
Mental Health Standard 5.1- By the end of grade 5, students identify the
various feelings that most people experience and describe the physical and
emotional reactions of the body to intense positive and negative feelings
Mental Health Standard 5.7- By the end of grade 8, students will identify and
describe the experience of different feelings (such as elation, joy, grief, and
rage) and how feelings affect daily functioning (MA Comprehensive Health
Curriculum Framework)
25% of the
measure
CCO: Student will identify healthy coping strategies.
Mental Health Standard 5.11- By the end of grade 12, students will analyze
healthy ways to express emotions and to cope with feelings, including the
common causes of stress, its effects on the body, and managing stress (MA
Comprehensive Health Curriculum Framework)
CCO: Student will implement healthy coping strategies to improve their
immediate emotional state.
Mental Health Standard 5.2: By the end of grade 5, students apply methods to
accommodate a variety of feelings in a constructive manner in order to
promote well being.
25% of the
measure
25% of the
measure
100%
1
1
A CCO is a statement that describes core, essential, or high priority content (knowledge, skills, or
abilities) identified by those who designed the assessment and drawn, synthesized, or composed from a
larger set of curriculum or professional standards. This Core Course Objective aligns with several
standards in the Massachusetts Comprehensive Health Curriculum Framework, as shown in the Content
Chart, above. These standards are designated for mastery by the end of grade 5, 8, or 12; all are relevant
throughout the middle and high school grades for those students who work directly with the school
psychologist because these students are typically identified for services due to challenges with these core
skills.
SEEM Collaborative – DDM – School Psychologists 6-12 3
Instrument
This DDM consists of three measures that provide information about students’
understanding and application of the specified social/emotional skills. The first is a
Student Survey, consisting of seven open-response questions to which the student
responds in writing at the start and end of the school year. This serves as a pre- and
post-measure and is intended to highlight growth over time. The second measure is a
set of four Journal Prompts, provided to the student in writing, as well as read aloud,
and to which the student then responds in writing on a monthly basis. The ongoing
journal prompts are based in a belief that writing is a form of reflection and provides
time for students to make sense and draw connections that oral assessments may not.
The third measure is a Student Interview Checklist, which is completed by the school
psychologist monthly based on the student’s oral response to a set of interview
prompts. This combination of assessments allows a robust view of growth over time by
bringing together several measures that include the observations of both student and
school psychologist.
Note that all three measures in this DDM contain items that address each content area
described in the Content Chart above. For example, items on the Student Survey
address each of the four content areas listed above; items in the student Journal
Prompts address each content area above; and items on the school psychologist
Interview checklist address each content area above. Point values for the various items
have been weighted, from five to nine points each, to reflect the proportions designated
in the chart.
Administration Protocol
The Administration Protocol addresses how the measure is intended to be implemented
to best support a common conversation about student growth across classrooms.
Which students should be included?
This measure is designed for school psychologists who work intensively with a small
caseload of three to five students who receive frequent, ongoing social/emotional crisis
intervention and support. If the psychologist has four or five students, all should be
included in this DDM. If the psychologist has more students on his or her caseload who
receive such ongoing, intensive social/emotional support, students should be randomly
selected by taking every other name from an alphabetical listing of last names of
students being serviced, up to a maximum of five students. For example, if eight
students are being serviced regularly by the school psychologist for social/emotional
skill intervention and support (listed here alphabetically by last name) – John Abel, Julie
Chase, David Clay, Fred Jones, Tai Moore, Carolina Naroni, Sonia Ramirez, and Tony
Romano - the psychologist would include every other student in this DDM – John Abel,
SEEM Collaborative – DDM – School Psychologists 6-12 4
David Clay, and Tai Moore, and Sonia Ramirez - or four in total. This process ensures
fair and unbiased selection of a limited number of students whose growth can be
tracked using multiple measures during the school year.
When is the measure administered?
Students complete the first measure, the Student Survey, during the first week of
service with the school psychologist (e.g., September) and once during the last week in
May or first week in June. The second measure, Journal Prompts, is administered to
the student at monthly intervals throughout the school year, beginning during the
second session with the school psychologist. The school psychologist administers the
third and final measure, the Student Interview, each month throughout the school year,
beginning during the third session with the psychologist. Each of the measures should
be administered during different counseling sessions so that the student’s engagement
in one activity does not overly influence the ways he or she engages in the others.
How is the measure administered?
Student Survey
This is a paper-pencil task for students. The student reads and answers each question
on the survey. The school psychologist may answer the student’s questions to clarify
the understanding of the assessment task; however, may not give examples of
appropriate answers. The school psychologist provides up to 20 minutes of quiet time
for the student to complete the task. If the student does not initiate the task, the school
psychologist may ask if the student needs clarification on the task or assesses whether
special accommodations are needed (see below).
In order to increase consistency in the administration of this assessment across school
sites, school psychologists should orally explain the following points to students:
1. This assessment will be given two times per year - at the beginning of the year,
before we have worked together in counseling sessions, and again toward the
end of the year after we have worked together.
2. The pre-survey portion of the assessment is designed to gather information on
what you already know. If you do not know the answers to some of the
questions-don’t worry. We will be learning about these topics throughout our
work together.
3. Your responses will help guide us on what we should work on throughout our
time together.
4. I will give you the survey. Your task is to read the survey and answer the
questions. If you have any question, you may ask me. I can clarify the task;
however, I am not able to provide you with specific examples on how to answer
questions.
5. You will have 20 minutes to work on the survey. If you do not finish in 20
minutes, don’t worry, I can provide more time if needed, but we need to finish
today and cannot continue the work into another session.
SEEM Collaborative – DDM – School Psychologists 6-12 5
6. After you hand in the survey, if I have questions about anything that you wrote, I
will ask you to clarify during the next time we meet.
7. This is not a test like you typically take in school. You will not be receiving a
grade on this task. Just try your best. I will keep your survey results and compare
them to your performance on the same assessment at the end of the year. This
will help me see how much you have grown throughout our sessions. It will also
help me to work better with students in the future.
8. I appreciate your honest answers.
Directions to Students, Post-Survey (last week of May/1st week of June):
1. I will use this post assessment to help me see what you have learned throughout
our work together. The post-assessment is exactly the same as the preassessment at the start of the year.
2. The skills, topics, conversations, and practice that we have completed throughout
our counseling sessions should help you feel prepared to answer these
questions.
3. Your responses will help me to provide effective counseling to students in the
future.
4. If you have any questions about the survey, you may ask me. I can clarify the
task; however, I am not able to provide you with specific examples on how to
answer questions.
5. You will have 20 minutes to work on the survey. If you do not finish in 20
minutes, don’t worry, I can provide more time if needed, but we need to finish
today and cannot continue the work into another session.
6. After you hand in the survey, if I have any questions about what you wrote, I will
ask you to clarify the next time we meet.
7. Remember, this is not a test like you typically take in school. You will not be
receiving a grade on this task. Just try your best. I will keep the results and
compare them to your performance on the same assessment that you took in
September. This will help me see how much you have grown throughout our
sessions. This will also help me to work better with students in the future.
Journal Prompts
The school psychologist reads the four journal prompts aloud to the student and
answers student-generated questions to clarify understanding of the assessment task;
however, may not give examples of appropriate answers. The prompt is also provided
to the student in a written format to refer to while writing. The school psychologist
provides 10-12 minutes to answer the journal prompts, so this is intended as a quick
write. If the student does not initiate the task, the school psychologist may ask if the
student needs clarification on the task or assesses whether special accommodations
are needed (see below).
In order to increase consistency in the administration of this assessment across school
SEEM Collaborative – DDM – School Psychologists 6-12 6
sites, school psychologists should orally explain the following points to students:
1. This assessment will be given every month during our counseling sessions.
2. Your responses will help me know what we should work on throughout our time
together.
3. Your responses will also help me monitor your growth throughout counseling.
4. I will read aloud the four journal prompts. I will then give you the prompts in
writing so you can review them on your own. Your task is to respond to each
prompt in writing because writing is a form of reflection and allows you some time
to think and connect ideas. If you have a question about the journal prompts, you
may ask me. I will be able to clarify the task, however, I will not be able to provide
you with specific examples on how to answer the prompt.
5. You will have 10-12 minutes to complete your writing. I can provide more time if
needed, but you will need to finish the writing today and cannot continue the work
into another session.
6. If I have questions about what you wrote, I will ask you to clarify the next time we
meet.
7. You will not be receiving a grade on this writing. Just try your best. I will compare
your writing to the journal writing you do each month. This will help me see how
you are growing throughout our sessions.
Student Interview Checklist
The school psychologist completes the interview checklist monthly during the student
counseling session. For this activity, the psychologist must have available an ageappropriate set of emotion cards to provide visual scaffolding for this language-based
activity, such as the following (a variety of emotion card resources are available online):
SEEM Collaborative – DDM – School Psychologists 6-12 7
and an “Intensity of Emotion” scale on a strip of cardstock that includes visual cues,
such as the following:
1
2
3
4
CALM
MILDLY
INTENSE
INTENSE
VERY
INTENSE
5
OVERWHELMED
It is strongly recommended that the visual cues on this Intensity Scale are described to
students in terms of intensity of emotion and are clearly distinguished from the emotions
themselves, such as anger, jealousy, sadness. To do this, school psychologists may
want to discuss and use both the visual emotion cards and the “Intensity of Emotion”
scale as a regular part of counseling sessions to engage students in recognizing and
naming their emotions, then describing the intensity of their emotional and physical
responses.
It is assumed that the psychologist using this DDM will often be intervening with
students based on a social/emotional need, so the Interview Checklist is designed as a
guided, reflective series of prompts to work through that crisis. The psychologist asks
the following series of six questions, providing up to 10 seconds of wait time for each
response.
1. What is your current emotion or feeling?
2. [Using the “Intensity of Emotion” scale] On a scale from 1 to 5, with 1 being
completely calm and 5 being very intense and overwhelmed, what is the intensity
of the emotion you are feeling right now?
3. How is your emotion or feeling affecting your body?
4. Name a healthy strategy for coping with this emotion.
5. Review the steps and try to use this coping strategy right now.
6. Using our scale from 1 to 5, what is the intensity of the emotion or feeling now?
SEEM Collaborative – DDM – School Psychologists 6-12 8
(If needed, what could help to further reduce the intensity of the emotion?)
The psychologist only provides additional prompts and support if the student explicitly
seeks clarification or support, or if the student is initially unable to respond after the
provided wait time. Adhering to the progression of possible student performances, listed
below and on the Interview Checklist, the psychologist provides one brief prompt or
clarification, if needed and as provided on the Checklist, then provides an additional 10
seconds of wait time for the student’s response. For example, if the student is unable to
name his or her current emotion, the psychologist may prompt, “Take another look
through the emotion cards. Is there one that shows how you are feeling today?” If after
this prompt and an additional 10 seconds of wait time the student is still unable to
respond, the psychologist may provide up to two additional prompts, but no more. The
psychologist is not allowed to provide suggested responses, such as, “You look very
angry today. Are you feeling angry?” Responses must always come from the student;
the psychologist’s task is to observe and learn from the student’s ability to respond.
In some cases, it may be possible that one or more of the particular questions is not
relevant to a given conversation with the student. For example, it may not be possible or
relevant for a student to try a particular identified copying strategy in the therapeutic
setting, in which case the psychologist would make note of this on the Student Interview
Checklist form by circling n/a (not applicable). In the vast majority of cases, however, it
is anticipated that it will be relevant for the student to respond to all six questions in the
Interview.
The psychologist records each of the student’s responses in the space provided on the
Student Interview Checklist, then circles the associated rating / point value that reflects
the demonstrated level of response:
o Yes, Demonstrated Independently
o Yes, Demonstrated with 1 Prompt
o Yes, Demonstrated with Multiple Prompts
o No, Did Not Demonstrate Despite Prompts
o No Opportunity to Demonstrate, Not Relevant
(Associated point values are differentiated according to the weighting shown in the
Assessment Blueprint.) The psychologist then adds the circled point values and records
in the final row of the chart as a total out of a possible total of 44 points.
How will accommodations be documented?
If the psychologist has observed or received information from teachers or student
evaluations that the writing demands of these assessments may not allow students to
convey their full understandings of the designated skills, the psychologist may
administer the Student Survey and Journal Prompts orally and scribe the student’s
oral responses. The psychologist may not provide any prompts, cues, or answers to the
student in the process, in accordance with the Administration Protocol above.
SEEM Collaborative – DDM – School Psychologists 6-12 9
In particular, the school psychologist must provide accommodations specified in
students’ Individualized Education or 504 Plans. In addition, for Stage 1 ELL students,
all instructions and assessment prompts must be translated – in writing and/or orally - to
the student’s native language to ensure that these students have opportunity to
understand the assessment tasks and demonstrate their understandings. Further, these
students must have the option to record their written responses for the Student Survey
and Journal Prompts in their native language and to speak responses to the Student
Interview in their native language if desired. It is strongly recommended that the
psychologist work closely with the school or district EL coordinator to provide these
accommodations and to translate and scribe these students’ responses back into
English for interpretation during scoring.
All accommodations provided, and the reasons for these accommodations, must be
documented on the provided scoring sheets for each of the three measures in this
DDM.
How are deviations to protocols addressed?
Students are randomly selected for this DDM at the start of the school year; however, if
fewer than three students suit the criteria for selection (e.g., work regularly and
intensively with the school psychologist on social/emotional skill development) at the
start of the school year, additional students may be added as the year progresses. The
pre-assessment Student Survey should be administered within two weeks of beginning
services with the school psychologist and monthly Journal Prompts and Student
Interview Checklist observations should begin during the two weeks following. If any
events disrupt the monthly administration of the Journal Prompts or Student Interview
Checklist (e.g., student or teacher absence, special school events), the school
psychologist should administer the measure at the next scheduled session with the
student with the aim of gathering progress data at approximately one month intervals.
Scoring Guide
The school psychologist should conduct and score all student work and performances
for this DDM because this person has the relevant professional knowledge to interpret
students’ demonstration of social/emotional skills evaluated in these assessments.
However, because this DDM is based on multiple one-to-one measures that occur
within the therapeutic setting, it is essential that school psychologists calibrate their
scoring with same-role colleagues in their school or in other schools in the district (e.g.,
school psychologists or school guidance counselors who work directly and intensively
with students on social/emotional skill development).
Calibration strengthens the likelihood that observations and scoring will be conducted in
a consistent and reliable manner over time and across sites. A sample calibration
SEEM Collaborative – DDM – School Psychologists 6-12 10
protocol can be found at the Rhode Island Department of Education website:
http://www.ride.ri.gov/Portals/0/Uploads/Documents/Teachers-and-AdministratorsExcellent-Educators/Educator-Evaluation/OnlineModules/Calibration_Protocol_for_Scoring_Student_Work.pdf
In particular, to calibrate scoring with this DDM, the school psychologist can bring
student Journal Prompts and pre- and post-Student Survey responses to a
collaborative session to calibrate scoring. The professionals may also aim to identify
anchor papers for each of these measures that represent different levels of performance
to guide and support common scoring over time. To calibrate scoring of the Student
Interview Checklist, the school psychologist will need to gather video footage of
several student interviews to use for a calibration exercise with colleagues.
How Should Scorers Prepare for Scoring?
To score each part, the school psychologist should make copies of the provided scoring
guides and set up meetings with colleagues to calibrate scoring. This is important to
make sure multiple school psychologists are able to score in the same manner, and
thereby contribute to fair and reliable scoring across a district.
How Should Assessment Scores Be Calculated?
Student Survey
The school psychologist consults the Student Survey Scoring Guide (Appendix D).
Using the scoring guidance and examples of possible acceptable responses, the
psychologist records the student’s name and assessment date at the top of the form,
evaluates each of the seven survey items, and circles the corresponding point values
indicating whether each student response is Fully Correct, Partially Correct, or Incorrect.
Finally, the school psychologist adds the seven scores and records the total in the
bottom right cell on the form.
Journal Prompts
The school psychologist consults the Journal Prompts Scoring Guide (Appendix E).
Using the scoring guidance and examples of possible acceptable responses, the
psychologist records the student’s name and assessment date at the top of the form,
evaluates each of the responses to the four journal prompts (and sub-components of
each prompt), and circles the corresponding point values indicating whether each
student response is Fully Correct, Partially Correct, or Incorrect. Finally, the school
psychologist adds the scores and records the total in the bottom right cell on the form.
The School Psychologist transfers total scores from Journal Prompt, Interview
Checklist, and Student Survey to the Score Tracking Sheet, then compiles and
calculates score from each month using the following steps:
1. Record the student name at the top of the Score Tracking Sheet.
2. Locate the appropriate month at the top of the form and record the total score for
SEEM Collaborative – DDM – School Psychologists 6-12 11
each assessment that was completed. The Student Survey should only be
completed in September and May. The Interview Checklist and Journal
Prompt should be completed once each month September through May. These
monthly scores allow the psychologist to track progress over time and to adjust
instructional strategies if progress is not evident, increasing the likelihood that
gains will be achieved over time.
3. For each month sub-total the points for the Interview Checklist and Journal
Prompt out of a total possible 96 points. For September and May, record the
total points for the Student Survey, Interview Checklist, and Journal Prompt
for a pre-test and post-test score out of a total possible 140 points.
How Should Gain or Growth Scores be Calculated?
Although the school psychologist will want to track growth for the Journal Prompts and
Interview Checklist from month to month during the year, final growth, or gain, scores
are formally calculated by totaling scores from all three measures at the start of services
(pre-test) and at the end of services in May (post-test).
After all three measures are completed in September, the scorer adds the results shown
in the first column on the Scoring/Tracking Sheet and records the total points out of
140 possible points for September. This process is repeated in May.
To calculate gains, the scorer subtracts the total score from September from the total
score from May. For example, if a student scored 50/140 points in September and
105/140 points in May, subtract 50 from 105. The gain score would be 55 points for this
particular student.
Measuring Growth and Settings Parameters
The following growth parameters were set with the understanding that the school
psychologist often works with several students every year who need crisis intervention
support related to self-management and application of social/emotional skills. Although
students’ personal situations will likely be varied, the ways the psychologist supports
their skill development may be somewhat similar, as reflected in the consistency of
items across the three measures in this DDM.
For this reason, growth parameters were set in terms of an overall pre- and post-test
gain score, calculated by adding results from all three measures in this DDM at the start
of the school psychologist’s services (pre-test score) and the end of services (post-test
score). An example is provided in Appendix F, which provides a Score Tracking Sheet.
We anticipate that students receiving services in grades 6-12 will have some basic
social/emotional skills, but that they have stalled in their development and, thus, need
more intensive support and intervention. Therefore, we expect the upper range of skills
may be more challenging for this particular group of students to achieve. For example,
SEEM Collaborative – DDM – School Psychologists 6-12 12
we anticipate that students will start the year scoring approximately 40-60 points and
that they will gain their first 20 or 30 points on this DDM more easily than their final 30
points. Therefore, the low growth parameters are broader than the high growth
parameters.
Low Growth
0 – 70 pts
Gain Score
Moderate Growth
71 – 110 pts
High Growth
111 – 140 pts
Piloting
These estimated growth parameters will be tested and refined during the pilot phase.
Given our short pilot period, we will pilot all three measures with several different grades
of students and gauge the span of skills between one grade and the next. Even though
these scores will not reflect individual growth, they will provide some measure of
average differences at different grade levels, providing an initial rough sense of one
year’s potential growth. For example, by gathering end-of-year scores from a sample of
7th graders and comparing their distribution and to a sample of 8th graders’ distributions
and mean score, we hope to gain an initial sense of what might be reasonable to expect
for one year’s growth on these measures. We will also take into consideration the
school psychologists’ observations and judgments of which students they believe made
low, moderate, and high growth during the year.
Our pilot plan is for the following educators, representing educators in and beyond the DDM
team at a range of schools, districts [and grades 6-11], to pilot this DDM between the end of
January 2015 and the end of May 2015:
Name
Role
School
District
Grade
Level
DDM Team
Primary Contact
1 Carly
Greenstein
School
Psychologist
North Reading
Middle School
North
Reading
6-8
Carly Greenstein
2 Sylvia Weisz
School
Psychologist
North Reading
Middle School
North
Reading
6-8
Carly Greenstein
3 Sarah Auger
School
Psychologist
Stoneham High
School
Stoneham
9-12
Sarah Auger
4 Colleen
Goode
Behavior
Specialist
Stoneham Public
Schools
Stoneham
k-12
Sarah Auger
SEEM Collaborative – DDM – School Psychologists 6-12 13
In particular, we aim to learn about the following questions from our pilot:
1. During counseling sessions, are students able to identify their feelings?
2. During counseling sessions, are students able to link how their feelings impact their
body?
3. During counseling sessions, are students able to identify healthy coping strategies?
4. During counseling sessions, are students able to apply coping strategies to improve their
current mood?
Development Process
This measure for School Psychologists was refined through both small trials and a longer term
pilot from January – June, 2015. During initial development, Carly Greenstein, Sylvia Weisz,
Sarah Auger, Colleen Goode conducted small tryouts of Journal Prompts, Interview Checklists,
and Student Surveys in North Reading Public Schools and Stoneham Public Schools. These trials
led to various refinements of measure’s items and descriptors, as well as to the scoring tools
and administration protocol. Due to the feasibility and repetitive nature of the multiple tools, it
was determined to solely use the Journal Prompt during the longer pilot period.
In addition, the assessment received multiple peer and expert reviews during initial
development. For example, staff at WestEd (Woburn, MA) critiqued the assessment task and
associated scoring materials, and an ESE review team also provided critical feedback, which led
to further revisions. For example, we changed the scoring of the instruments and the growth
parameters.
Finally, a longer term pilot of the assessment in the developers’ own schools and districts will
be conducted during the 2015-2016 school year to contribute to the development of increasing
specific, feasible, and informative tools for potential users.
Pilot Results
Initial feedback was positive. Those administering the pilot thought that the measures would be
feasible throughout counseling sessions. However, throughout the pilot phase the feedback
obtained noted that the measures were repetitive in nature. The student journal was
recommended as the most beneficial tool. Data patterns and trends over time will be collected
throughout the longer pilot phase during the 2015-2016 school year.
Example Data Table
Journal Prompt
Interview Checklist
Subtotal
Student Survey
Total
April
0/52
9/44
9/96
30/44
39/140
May
Gain
18/52
10/44
28/96
37/44
65/140 26
SEEM Collaborative – DDM – School Psychologists 6-12 14
Feedback
Student
Survey
Seemed to be better for
younger students. It took
approximately 10 mins. to
administer.
Journal
Prompt
Responses seemed to be based on
students' current emotional state. Is this
an accurate assessment of their ability
to complete the task, or, an assessment
of how they feel in the moment?
Student
Interview
Checklist
Format seemed to
be too formal for
counseling
sessions.
At times, awkward to
administer to high
school students.
Students felt it was
juvenile.
Weird to do so close
Some questions
together but would
too complex for
make sense beginning 6th graders
and end of year
Seemed to be
tailored for
younger
participants.
Did not seem to be
helpful. Were not
applicable to
counseling sessions.
Hard for
some
students
Student didn't know
what the words
implementing a coping
strategies meant
Was not applicable.
Sessions scheduled at
certain time and student
usually in a good mood.
Didn't make sense
most of the time
unless student
comes in crisis
In light of the pilot results, we decided to eliminate the interview checklist and student survey.
The team determined that the journal prompts allowed students to reflect and report on their
current emotion, how their emotion impacted their body, as well as possible coping strategies
to improve their overall emotional state. This measure is useful throughout 6-12th grade, is age
appropriate, and can be conducted in counseling sessions throughout a school year. This will
increase the school psychologist's opportunity to monitor student growth in a feasible and
measurable manner. The next steps include solely using the journal prompts as a single
measure to assess students’ growth within counseling sessions. Therefore, scoring parameters
will need to be changed in order to accurately measure this DDM. However, we do not feel that
we have enough information to recalibrate scoring at this time. A longer term pilot is
recommended in order to collect data, increase sample size, and improve overall scoring.
If the interview checklist and student survey were to be used, the recommendation of the pilot
group would be to amend these items to allow for a wide-range of student emotions at the
time of the administration of the survey as this would lead to more practical uses of the tool.
Assessment Blueprint
The assessment blueprint is an elaboration of the content table included in the
introduction. It serves two purposes: (1) it is a roadmap for the assessment
development team to ensure balanced coverage of the most important content and (2) it
is a key for other potential users of the assessment by concisely indicating what content
the assessment is designed to measure and the level of rigor with which the content is
covered (see pages 12 and 29 of Technical Guide A for more information).
SEEM Collaborative – DDM – School Psychologists 6-12 15
The DDM developers opted to approximate the total weight of each content area, with
35 items representing approximately 25% of the DDM score in each of the first two
content areas and 36 items representing approximately 25% of the DDM score in each
of the remaining two content areas.
SEEM Collaborative – DDM – School Psychologists 6-12 16
Appendix A
Student Survey
Name: _____________________________
Date: ______________________
1. Susie was not invited to her classmate’s birthday party over the weekend.
a. What emotion is she feeling?
b. Explain why you chose this emotion.
2. Bob got 100% on a test that he studied really hard for.
a. How is he feeling?
b. Explain why you chose this emotion.
3. Consider how you are feeling right now.
a. What emotion are you feeling?
b. Explain why you are feeling this emotion.
4. How is your body reacting to the emotion that you are feeling right now?
5. Johnny is feeling really nervous about a class presentation.
a. How might his body be reacting?
6. Judy is very angry because her parents will not let her go to the movies with her friends.
a. How might her body be reacting?
7. Consider the healthy coping strategies you use.
a. What healthy coping strategy do you often use to help yourself feel better?
b. How does your body react when you use this strategy?
(Continue writing on back)
SEEM Collaborative – DDM – School Psychologists 6-12 17
Appendix B
Student Journal Prompts
1. Think of a situation in the past two weeks when you experienced an unpleasant
emotion.
a. Describe the situation.
b. Describe the emotion you felt.
c. Describe how your feeling affected your body.
2. Think back to the coping strategy that you used to improve this situation.
a. Name the coping strategy and explain the steps that you used.
3. Explain why you chose to use this particular coping strategy.
4. Describe how the coping strategy helped.
OR If the coping strategy did not help, describe what other healthy coping strategy
you ci use if this situation happened again.
SEEM Collaborative – DDM – School Psychologists 6-12 18
Appendix C
Student Interview Checklist (for Teacher Use)
Adult Prompt
Acceptable Response
1. What is your current emotion or
feeling?
Notes:
Student labels current state with a
relevant emotion word (e.g., sad,
angry, frustrated)
If needed, first prompt: Take
another look through the emotion
cards. Is there one that shows how
you are feeling today?
2. On a scale from 1 to 5, with 1
being completely calm and 5 being
very intense and overwhelmed,
what is the intensity of the emotion
you are feeling right now?
Notes:
Student identifies intensity of
feeling on a provided scale from 1
to 5.
If needed, first prompt: Remember,
the first picture shows a calm face
because the emotion is very mild
at this end of the scale. Each
picture shows a little more intensity
to the emotion. At this end, you
would be having a very hard time
controlling your emotions because
they would feel super intense and
overwhelming. Where on the scale
would you say you are feeling right
now?
3. How is your emotion or feeling
affecting your body?
Notes:
Student links current feeling to one
or more physical symptoms that
typically occurs in concert with the
emotion identified.
Yes
Demonstrated
Independently
Yes
Demonstrated
with 1 Prompt
Yes
Demonstrated
with Multiple
Prompts
No
Did Not
Demonstrate
Despite
Prompts
No
Opportunity to
Demonstrate
Not Relevant
5
4
2
0
n/a
5
4
2
1
n/a
7
5
3
0
n/a
SEEM Collaborative – DDM – School Psychologists 19
If needed, first prompt: Think about
how the inside of your body feels
right now - your muscles, your
heart beat or pulse, your hands
and legs, your head, neck and
shoulders. Where in your body do
you feel your emotions right now?
Can you describe how your
emotions are making your body
feel?
4. Name a healthy strategy for
coping with this emotion.
Notes:
Student identifies a coping
strategy that would normally
reduce the intensity of the
identified emotion and support
productive interactions.
If needed, first prompt:
What strategies can you recall that
we have worked on together this
year? Would any of these be
helpful for coping with the emotion
you are feeling right now?
5. Review the steps and try to use
this coping strategy right now.
Notes:
Student tells the steps and applies
the coping strategy in the moment.
If needed, first prompt: What is the
very first step you need to take to
use this strategy? What do you
need to do after that? Keep going,
take me through the steps. Then
try the steps.
6. Using our scale from 1 to 5,
what is the intensity of the emotion
or feeling now? (If the emotion is
not calmed down yet, ask, “What
Student indicates that the intensity
of the emotion has decreased and,
if needed, describes steps that
could further reduce the intensity
9
6
3
0
n/a
9
6
3
0
n/a
9
6
3
0
n/a
SEEM Collaborative – DDM – School Psychologists
20
could help to further reduce the
intensity of the emotion?”)
Notes:
of the felt emotion.
If needed, first prompt: Remember,
the first picture shows a calm face
because the emotion is very mild
at this end of the scale. Each
picture shows a little more intensity
to the emotion. At this end, you
would be having a very hard time
controlling your emotions because
they would feel super intense and
overwhelming. Where on the scale
would you say you are feeling right
now?
TOTAL SCORE
Special Accommodations Provided:
/ 44
Rationale for Accommodations Provided:
SEEM Collaborative – DDM – School Psychologists
21
Appendix D
Student Survey Scoring Guide (For Teacher Use)
Fully Correct Responses:
- Answers all parts of the question and - Provides relevant and plausible responses for all parts
Partially Correct Responses: - Answers some parts of the question or - Answers all parts, but not always relevant or plausible responses
Not Correct Responses:
- Answers are incomplete
and/or - Answers are not relevant or plausible
Student Name:
#
1
2
3
4
5
6
7
Question
Date:
Guidance
Susie was not invited to her classmate’s
birthday party over the weekend.
- What emotion is she feeling?
- Explain why you chose this emotion.
Student identifies how he/she
might feel and explains why this
feeling might be likely after being
left out of event by a peer
Bob got 100% on a test that he studied
really hard for.
- How is he feeling?
- Explain why you chose this emotion.
Student identifies a positive
feeling related to
accomplishment
Consider how you are feeling right now.
- What emotion are you feeling?
- Explain why you are feeling this emotion.
Student names an actual feeling
or emotion, rather than a
description, an example, a wish,
or other response
Student describes feeling in the
body that connects with the
listed emotion
Student describes a feeling
related to an undesired activity
How is your body reacting to the emotion
that you are feeling right now?
Johnny is feeling really nervous about a
class presentation.
- How might his body be reacting?
Judy is very angry bc her parents will not let
her go to the movies with her friends.
- How might her body be reacting?
Consider healthy coping strategies you use.
- What healthy coping strategy do you often
use to help yourself feel better?
- How does your body react when you use
this strategy?
Special Accommodations Provided:
Examples of Correct
Responses
Sad, mad, lonely
Susie would probably feel angry
because it is not kind to leave
friends out of a special event.
Proud, excited, happy
Bob would feel proud because
he probably studied really hard
to get every question right.
Fully
Correct
Partially
Correct
Not
Correct
5
2
0
5
2
0
5
2
0
7
3
0
7
3
0
Angry, frustrated, sad
Angry: Trembling, pit in stomach
Sad: Lack of energy
Tense: I have a headache
Sweating, racing heart, stomach
ache
Student describes physical
symptoms related to anger
Tense, hot, pit in stomach,
verbal outburst, shaking
7
3
0
Student identifies a healthy,
productive coping strategy and
describes how his/her body
reacts in response
Go for walk: calms my mood
Breathe deeply: relax muscles
Count to 100: distracts me
8
4
0
TOTAL SCORE
Rationale for Accommodations Provided
/44
SEEM Collaborative – DDM – School Psychologists 22
Appendix E
Journal Prompts Scoring Guide (For Teacher Use)
Fully Correct Responses:
- Answers all parts of the question and - Provides relevant and plausible responses for all parts
Partially Correct Responses: - Answers some parts of the question or - Answers all parts, but not always relevant or plausible responses
Not Correct Responses:
- Answers are incomplete
and/or - Answers are not relevant or plausible
Student Name:
Item
#
Date:
Question
Prompt: Think of a time in the past two
weeks when you experienced an
unpleasant feeling or emotion.
a. Describe the situation.
b. Describe the emotion you felt.
1
c. Describe how your feeling
affected your body.
2
Prompt: Think back to the coping
strategy that you used to improve this
situation
a. Name a coping strategy and the
steps that you used.
Guidance
Examples of Correct
Responses
Acceptable?
Fully
Correct
Partially
Correct
Not
Correct
.
Student describes an unpleasant
situation and what happened.
Student correctly identifies an emotion
using an emotion word rather than a
description of what the emotion was
“like” or what the student wanted to do
in response to the emotion.
Student links feelings with physical
state, suggesting connection between
emotion and muscles, sweat or tear
glands, tensions (headache, neck or
shoulder cramps, jittery hands or
legs), heart rate, pulse, or energy
level, etc.
Student correctly names a taught
coping strategy and explains a correct
and complete sequence of steps to
Student identified an unpleasant
experience within the academic,
social, and/or personal realm.
Sad, Anxious, Angry, Jealous,
Depressed, Furious, Confused
5
5
Lack of energy (sad)
Sweaty palms, racing heart
(anxious, nervous)
Tense, hot, pit in stomach
(angry, mad)
Deep breathing.
Be aware of my breathing.
2
2
0
0
6
3
0
9
4
0
SEEM Collaborative – DDM – School Psychologists
23
3
4
implement it.
2. Slow down my breathing.
3. Breathe more deeply and
longer.
4. Feel my heart beat.
Prompt: Explain why you chose to
use this particular coping strategy.
Student explains at least one
plausible reason for choosing a
specific strategy.
I felt my heart begin to race so I
tried to slow my breathing down
and take deeper breaths.
Prompt: Describe how the coping
strategy helped.
Student describes how the coping
strategy helped to address the
negative emotion and situation.
Felt calmer so didn’t react
strongly or immediately to the
other person and was able to
walk away or respond calmly.
Prompt: OR if the coping strategy did
not help, describe what other healthy
coping strategy you could use if this
situation happened again.
Student identifies how the strategy
failed to improve current emotional
state and also identifies an alternative
healthy coping strategy to use for the
future.
Next time, try to pay attention to
the triggers that are upsetting
me, such as the person’s use of
blaming language.
Total
Special Accommodations Provided:
9
4
0
9
4
0
9
4
0
/52
Rationale for Accommodations Provided:
SEEM Collaborative – DDM – School Psychologists
24
Appendix F
Scoring Tracking Sheet
Student: ______________________
Journal
Prompts
Interview
Checklist
Subtotal
Student
Survey
Total
Sept
Oct
Nov
Dec
Jan
Feb
March
April
May
Date:
/52
Date:
/52
Date:
/52
Date:
/52
Date:
/52
Date:
/52
Date:
/52
Date:
/52
Date:
/52
/44
/44
/44
/44
/44
/44
/44
/44
/44
/96
/44
/96
/96
/96
/96
/96
/96
/96
/96
/44
/140
Gain
Date:
/140
Example
Journal
Prompts
Interview
Checklist
Subtotal
Student
Survey
Overall
Total
Sept
Oct
Nov
Dec
Jan
Feb
March
April
May
Date:
7/52
Date:
13/52
Date:
13/52
Date:
17/52
Date:
26/52
Date:
17/52
Date:
35/52
Date:
41/52
Date:
41/52
Gain
Date
+34 pts
10/44
10/44
17/44
10/44
17/44
26/44
26/44
35 /44
44/44
+34 pts
17/96
16/44
23/96
30/96
27/96
43/96
43/96
61/96
76/96
85/96
32 /44
+68/96
+16 pts
117/140 +84/140
points
33/140
This student’s overall gain for the year +84 points.
SEEM Collaborative – DDM – School Psychologists 25