Waiting Room
Today’s webinar will begin shortly.
REMINDERS:
 Dial 800-503-2899 and enter the passcode
6496612# to hear the audio portion of the
presentation
 Download today’s materials from the sign-in
page:
 Webinar Series Part 7 PowerPoint slides
 Using Current Assessments in DDMs
Massachusetts Department of Elementary & Secondary Education
Ramping Up for Next Year:
Strategies for Using Current
Assessments in DDMs
Webinar Series Part 7
DDM and Assessment Literacy Webinar Series
#
Date
Length
Time
3/14
60 minutes
4-5pm
4/4
4/25
7/11
90 minutes
60 minutes
3 hours
4-5:30pm
4-5pm
9am-12pm
7/18
60 minutes
4-5pm
8/15
60 minutes
4-5pm
TA and Networking Session II
9/19
3 hours
2:30pm5:30pm
6
Integrating Assessments into Educator
Evaluation: Developing Business Rules and
Engaging Staff
10/24
60 minutes
4-5pm
7
Ramping up for Next Year: Strategies for
Using Current Assessments in DDMs.
12/5
60 minutes
4-5pm
TA and Networking Session III
12/12
3 hours
Communicating Results
Sustainability
1/23
2/27
60 minutes
60 minutes
1
2
3
4
5
8
9
Title
Introduction: District-Determined Measures and
Assessment Literacy
Basics of Assessment
Assessment Options
TA and Networking Session I
Determining the Best Approach to DistrictDetermined Measures
Measuring Student Growth and Piloting DistrictDetermined Measures
2:30pm5:30pm
4-5pm
4-5pm
Audience & Purpose
Target audience
District teams that will be engaged in the work of
identifying, selecting, and piloting DistrictDetermined Measures.
After today participants will understand:
Strategies for using current assessments as a
starting place for District Determined Measures
3
Massachusetts Department of Elementary & Secondary Education
Educator Evaluation Data
SY 2012-2013
Data Release – 11/21/13
2012-13 educator evaluation performance
rating data released for 213 RTTT districts
37,940 educators
Majority of Educators Received Solid Ratings
85.2% Proficient
Early evidence of differentiation is encouraging
7.4% Exemplary; 6.8% Needs Improvement; and
0.7% Unsatisfactory
Massachusetts Department of Elementary & Secondary Education
5
Ramping Up for DDM
Implementation
Identifying DDMs beyond the minimum pilot areas
Key Messages for Stakeholders
7
Massachusetts Department of Elementary & Secondary Education
Ramping Up
Goal by June 2014: To identify at least two
measures of student growth to match with
each educator for use in the 2014-15 school
year.
8
Massachusetts Department of Elementary & Secondary Education
Ramping Up
Strategies:
Collaborate with other districts (e.g. network at the
upcoming TA sessions, work with area
collaboratives)
Engage educators in the process of identifying
DDMs
Use assessments currently in use in the district as a
starting point for DDMs
9
Massachusetts Department of Elementary & Secondary Education
Why use Current Assessments?
Feasibility
Potentially cost effective
Familiarity and experience with the assessments
They are Aligned to Content
Assess what is most important for students to learn
Assess what the educators intend to teach
They are Informative
Teachers already use the assessments to measure student
learning
Final Step
Provide valuable information to schools and districts about
an educator’s impact
Massachusetts Department of Elementary & Secondary Education
10
Considerations in DDMs
Performance vs. Growth
 Most current assessments are measures of performance and
not growth
 Growth takes into account the different levels of student
achievement
 Measures of growth should provide all students an equal
opportunity to demonstrate growth
Unit vs. Year
 Most current assessments are designed to assess short units
 DDMs that represent a full year provide more accurate
representations of student growth and of an educator’s
impact
Massachusetts Department of Elementary & Secondary Education
11
Potential Solutions
Bring together multiple assessments of similar
content to measure performance at different
points in time
Administer one assessment (or similar
versions) at multiple points during the year
Combine multiple assessments of growth into
a single DDM (e.g. pre-post tests of multiple
units)
Massachusetts Department of Elementary & Secondary Education
12
The larger DDM context
Preparing
Step
1. Create a Team
Key Questions


2. Determine Content to Be 
Measured
Who will be on the team?
What are the key responsibilities?
What standards and objectives are the most important to measure?
3. Identify Measure


What measures are already in use in the district?
Do components need to be developed or modified before piloting?
4. Prepare to Implement
Measure



What is the timeline?
How will measure be scored?
How will the data be collected and stored?
Implementing
Step
Key Actions
5. Test



Administer the measure
Score the results
Collect the data
6. Analyze


Analyze student results
Analyze the administration and scoring processes
7. Adjust

Modify components as needed
8. Repeat

Test, analyze, and adjust
Massachusetts Department of Elementary & Secondary Education
13
New Guidance from ESE
Using Current Assessments in District-Determined
Measures: Leveraging the Curriculum-Embedded
Performance Assessments From the Model
Curriculum Units
 Available at
http://www.doe.mass.edu/edeval/ddm/UsingAssessments.pdf
14
14
Massachusetts Department of Elementary & Secondary Education
Using Current Assessments:
Three Steps
1. Identify key content.
2. Ensure that change in performance
represents student growth.
3. Select an approach for measuring growth.
15
Massachusetts Department of Elementary & Secondary Education
Using Current Assessments:
Three Steps
1. Identify key content.
2. Ensure that change in performance
represents student growth.
3. Select an approach for measuring growth.
16
Massachusetts Department of Elementary & Secondary Education
Step 1: Identify Key Content
Key Content may include content that is:
A meaningful sample of the learning
Challenging to teach and learn.
Under represented in assessment
District priorities.
17
Massachusetts Department of Elementary & Secondary Education
Example: 3rd Grade ELA
Looking across the curriculum map to identify
a meaningful sample of content
18
Massachusetts Department of Elementary & Secondary Education
Example: 3rd Grade ELA
Model Curriculum Unit
Name
Writing Standards Covered in Each MCU
1
2
3
4
Independent Readers
Extreme Weather
6
7
8
9
10
X
Author Study
Biography
5
X
X
X
X
X
X
X
X
X
X
Newspaper
X
Poetry
X
Reading Inquiry
X
X
X
X
19
Stories Matter
Whose Story
X
X
Massachusetts Department of Elementary & Secondary Education
Example: 3rd Grade ELA
Standard 3 reads (W3.3) Write narratives to
develop real or imagined experiences or
events using effective technique, descriptive
details, and clear event sequences.
20
Massachusetts Department of Elementary & Secondary Education
Example: 9th Grade US History
Are there commonalities in content covered
across multiple units aligned to the MA
Frameworks.
21
Massachusetts Department of Elementary & Secondary Education
Example: 9th Grade US History
Model Curriculum Unit Name
Standards
Constitutional Rights
ELA9-10 W.1: Write arguments to support claims in an
analysis of substantive topics or texts, using valid
reasoning and relevant and sufficient evidence.
Economies of North & South
RH 10.1: Cite specific textual evidence to support analysis
of primary and secondary sources, attending to such
features as date and origin of the information.
WHST 10.9: Draw evidence from informational texts to
support analysis, reflection and research
Industrial Revolutions
W.H. 6-8.1 b: Support claims with logical reasoning and
relevant, accurate data and evidence that demonstrate an
understanding of the topic or text, using credible sources.
Westward Expansion
HSS Writing.9-10-1: Introduce precise claim(s), distinguish
the claim(s) from alternate or opposing claims, and create
an organization that establishes clear relationships among
the claim(s), counterclaims, reasons, and evidence.
Massachusetts Department of Elementary & Secondary Education
22
Step 1: Identify Key Content
Key content may be taught repeatedly across
the year
Key content may be taught once during the
year
23
Massachusetts Department of Elementary & Secondary Education
Step 1: Identify Key Content
Question for Engaging Educators
What can a student do after they take your
class that they couldn’t do before?
24
Massachusetts Department of Elementary & Secondary Education
Using Current Assessments:
Three Steps
1. Identify key content.
2. Ensure that change in performance
represents student growth.
3. Select an approach for measuring growth.
25
Massachusetts Department of Elementary & Secondary Education
Step 2: Does change represent growth?
What is Growth?
Educators understand growth.
Growth is about improvement and learning.
There is a difference between growth and how
we measure growth.
We can measure growth in both simple and
sophisticated ways.
26
Massachusetts Department of Elementary & Secondary Education
Step 2: Does change represent growth?
Are the assessments similar enough to support
meaningful inferences about student growth
during the year?
Do early assessments provide meaningful
information about what students do not
understand?
Do future assessments provide meaningful
information about what students have learned?
Do students have the opportunity to demonstrate
different levels of growth?
Massachusetts Department of Elementary & Secondary Education
27
Example: 3rd Grade ELA
In the third grade example
Each unit has a CEPA covering the same content
Rubrics are different, hard to compare
Solution
Identify similar evidence
Modify rubrics by adding consistently worded rubric
items that assess the same content
28
Massachusetts Department of Elementary & Secondary Education
Unit Name
Author Study
Rubric Items
1
2
3
4
5
6
Topic
development
and clarity
Evidence and
Content
Accuracy
Organization
Standard
English
Conventions
Variety of
Sentences
Precise Use
of
Vocabulary
Organization:
relevant, well
organized
and detailed
Opinion:
Includes
persuasive
details that
support the
writer’s
opinion
Illustrations:
Includes
engaging
illustrations
on each page
Text
Features:
Includes text
features
Voice: Turns
the events
into a story
The Writing
Process
Use of
Technology
Language
and
Conventions
Interviewing
and Quotes
Descriptive
Details
Includes
description of
topic of
poem
Includes
senses or
emotions
evoked by
poem
Specific
words or
phrases to
support
response
Personal
reflection
Topic
development
Evidence and
Content
Accuracy
Illustrations
Standard
English
Conventions
Looking Across Assessments
Biography
Newspaper
Poetry
Whose Story
Organization
Massachusetts Department of Elementary & Secondary Education
29
Example: 3rd Grade ELA
Conclusion:
The rubrics are similar enough to conclude that
change in performance represents growth.
Advantages:
Do not need to change the tasks students will
complete
Rubrics are on a consistent scale
Rubric do not need to be identical
Still include important rubric items that are present in only
one CEPA (for example, use of technology in the
Newspaper Unit)
Massachusetts Department of Elementary & Secondary Education
30
Example: 9th Grade US History
 Constitutional Rights: Presentation where
students take on different roles
 Economies of North & South: Write a journal,
letter, … etc.
 Industrial Revolutions: Persuasive Writing
 Westward Expansion: Work in small groups
making exhibit
31
Massachusetts Department of Elementary & Secondary Education
Example: 9th Grade US History
Conclusion:
The performance tasks are too dissimilar to
conclude that change in performance represents
growth.
Next Steps:
Identify a way to assess the key content
consistently.
For example, administer a single assessment of the
key content at multiple times during the year.
32
Massachusetts Department of Elementary & Secondary Education
Using Current Assessments:
Three Steps
1. Identify key content.
2. Ensure that change in performance
represents student growth.
3. Select an approach for measuring
growth.
33
Massachusetts Department of Elementary & Secondary Education
Step 3: Select an approach
Think about the scoring approach that will
best capture student learning given the type of
measure you have chosen.
For example:
Pre-Test/Post Test
Repeated Measures
Holistic Evaluation
Post-Test Only
Learn more
Webinar 5 http://www.doe.mass.edu/edeval/ddm/webinar.html
Technical Guide B http://www.doe.mass.edu/edeval/ddm/TechnicalGuide-AppxB.pdf
Massachusetts Department of Elementary & Secondary Education
34
Example: 3rd Grade ELA
Each student has five time points.
Classroom teacher rates all five time points,
which provides information to parents and
teachers about growth across the year.
First and last assignments are rated by two
additional teachers. The difference is used to
create a growth score. (The sum of the four
rubric items. They decide to set standards
after the pilot year.)
Massachusetts Department of Elementary & Secondary Education
35
Example: 9th Grade US History
Each student completes four writing
assignments.
The two US History teachers create a holistic
rubric that looks at four aspects of the writing.
The two US History teachers flag
disagreements between ratings and meet to
resolve these differences.
36
Massachusetts Department of Elementary & Secondary Education
More Information
Using Current Assessments in District
Determined Measures: Leveraging the
Curriculum-Embedded Performance
Assessments From the Model Curriculum Units
Available at
http://www.doe.mass.edu/edeval/ddm/UsingA
ssessments.pdf.
37
Massachusetts Department of Elementary & Secondary Education
Closing
Attend Next Week’s TA Session
 Date: December 12, 2013
 Time: 2:30pm – 5:30pm
 Locations: Southeastern MA: Norwell High School,
Little Theatre, 18 South Street, Norwell, MA 02061
 Northeastern MA: Groton-Dunstable Regional High
School, 703 Chicopee Row, Groton, MA 01450
 Western MA: Frontier Regional School, Media Center,
113 North Main Street, South Deerfield, MA 01373
 Register:
http://www.surveygizmo.com/s3/1462056/Registration
-DDM-Technical-Assistance-Networking-Session-3December-12th-2013
Massachusetts Department of Elementary & Secondary Education
39
Sharing Locally-Developed DDMs
We need your help! Please submit:
Assessment directions, materials, items and
prompts
Scoring resources (rubrics, answer keys)
Other information (steps for administration, lesson
plans, examples of how results are used)
To submit your examples, complete the survey
and upload your materials at
https://app.smartsheet.com/b/form?EQBCT=5
2e3762ac7954f93aa40cc879bc38855.
Massachusetts Department of Elementary & Secondary Education
40
Register for
Webinar Series Part 8
Part 8: Communicating Results
Date: January 23, 2014
Time: 4-5pm EST (60 minutes)
Register: https://air-event500.webex.com/airevent500/onstage/g.php?d=596783122&t=a
41
Massachusetts Department of Elementary & Secondary Education
Questions
Contact
Craig Waterman at cwaterman@doe.mass.edu
Ron Noble at rnoble@doe.mass.edu
Tell us how we did:
http://www.surveygizmo.com/s3/1469270/Dist
rict-Determined-Measures-AssessmentLiteracy-Webinar-7-Feedback
42
Massachusetts Department of Elementary & Secondary Education