District-Determined Measure Example
Teacher Teams’ Use of Data to Support Student Learning
Content Area and Grade Range: Administrators / School Leaders, K-12
DDM Summary: This DDM assesses administrators’ indirect impact on student learning by
measuring growth in teachers’ use of student data during their common planning meetings to
identify the varied needs of their students and to inform their interventions and instruction.
Developed by: James Pignataro, High School Principal (Grafton Public Schools), Timothy Fauth,
Middle School Principal (Grafton Public Schools), and Donna Dankner, Elementary Principal
(Maynard Public Schools)
Date updated: June 2015
Table of Contents
Introduction ............................................................................................................................ 2
Instrument .............................................................................................................................. 5
Administration Protocol ........................................................................................................ 6
Scoring Guide ...................................................................................................................... 10
Measuring Growth and Setting Parameters ....................................................................... 14
Piloting .................................................................................................................................. 14
Assessment Blueprint ......................................................................................................... 14
Assabet Valley Collaborative – DDM – Teacher Teams Use of Data 1
Introduction
Description of the Measure & Rationale
The purpose of this indirect measure is to provide timely feedback to school leaders about their
impact on school-wide data use for instructional purposes during teachers’ common planning time
meetings. The central measure is a 14-item Team Meeting Data Use Survey to be completed by
each common planning team (or if time is limited, by a designated member of the team) at the close
of two meetings per month throughout the school year. The survey requires less than five minutes
to complete and solicits team perceptions of the extent to which their team engaged in data analysis
to identify students’ strengths and needs and to determine how best to advance students’ learning.
School administrators are charged with providing teachers with common planning time and supports
for using this time effectively, especially in relation to data inquiry cycles and use of assessment and
other student learning data. This DDM provides administrators with frequent feedback from common
planning teams about the types of data they are discussing, the depth of their data use, the
conclusions and actions that occur as a result of their data use, as well as the supports the team
feels would help to advance their data use.
Intended use of the measure
To show growth over time, administrators are expected to directly observe teams using data and to
reflect on their own priorities for collaborative data use in the school, as a means of more fully
understanding the feedback provided through this measure. Administrators will need to consider the
ways and the extent to which they provide support to teams, through direct feedback and the
provision of tools, resources, and professional learning. In addition, administrators will need to
consider how their current practices may even undermine teams’ progress. For example, results of
this measure may indicate that teams spend their meetings discussing school- or district-assigned
tasks or specific matters that teams propose are important to their work, but do not involve data use.
In order to demonstrate growth on this measure and improve effective data use across the school’s
teams, administrators may need to revisit their own priorities and communications with teams, as
well as the allocation of time for teachers to accomplish tasks that are either required of them or
deemed important to their work.
Teachers’ teams across a school – and certainly across schools in a district – are often quite varied
in their ability to use a range of data effectively to support and advance students’ learning.
Therefore, this DDM is designed as a growth measure to support continuous improvement over
time, rather than as a target measure where the goal is to maintain a previously achieved
performance level. This DDM may be modified to serve as a target measure when school-based
teams begin to demonstrate high levels of data use to inform decisions about programs, student
interventions, and classroom instruction.
Context & Modifications for Other Contexts
This DDM was designed for use in small- and mid-sized districts, although the tools and processes
might be used in larger districts, as well. This DDM relies on honest and meaningful survey
feedback and shared commitment to developing effective data use in teams. It is thus
Assabet Valley Collaborative – DDM – Teacher Teams Use of Data 2
recommended that any district adopting this DDM consider the current level of trust between
administrators and teachers, as well as between central office and school leaders. Administrators
should also consider the frequency, structure, and priorities already established for teachers’
common planning time meetings and how this DDM may need to be modified to align with local
priorities. The Team Meeting Data Use Survey should be modified to reflect local data sources, and
the administration protocol should be modified to reflect the frequency of local team meetings. For
example, the current protocol indicates that teacher teams should complete the survey at two
meetings per month; in another district, however, teams may meet more or less frequently.
Description of the Development Process
This DDM was developed during October 2014 – June 2015 under a DDM Leadership Grant (FC217) awarded to the Assabet Valley Collaborative by the Massachusetts Department of Elementary
and Secondary Education (ESE). In partnership with Teachers21, Risk Eraser, and with primary
support from the Learning Innovations Program at WestEd (Woburn, MA), the Collaborative
convened six building administrators – three principals and three assistant principals – representing
schools spanning grades pre-K through 12 from five participating districts. Participants worked in
smaller teams of one to three to strengthen and apply their assessment literacy toward the
development of several direct and indirect measures of student growth.
Participants grew their expertise over six sessions by engaging in a guided DDM development
process framed by a series of questions: (1) Why are we doing this work together? (2) What is most
important to measure? (3) How shall we measure what’s most important? (4) How can we
strengthen and refine our measure? (5) How can we prepare our measure for broader use? (6) How
will we measure growth? Throughout, participants engaged in large group discussion and critique,
as well as team collaboration and problem solving.
This measure has not yet been piloted. Districts in and beyond the Assabet Valley Collaborative
may decide if they would like to pilot and/or modify the attached tools and processes for use as a
DDM in their district. Because this is a newly developed measure, it is important that districts
engage administrators in examining results from the first year of implementation. It is also important
to identify, over time, any revisions or refinements that may further strengthen the quality of the
assessment, scoring tools, administration protocol, and/or growth parameters to suit the
circumstances and realities of each district’s local context.
Content Alignment & Rationale
This measure is aligned to the following Core Objective (CO)1:
Educators regularly analyze a range of student assessment data to identify students’ strengths
and needs and to determine interventions and adjustments to instructional practice that will
ensure all students progress in their learning.
1
A Core Objective is a statement that describes core, essential, or high priority content (knowledge, skills, or abilities)
that was identified by those who designed the assessment and was drawn, synthesized, or composed from a larger set
of curriculum or professional standards.
Assabet Valley Collaborative – DDM – Teacher Teams Use of Data 3
This Core Objective (CO) was identified as the basis for this DDM because of the central role that
data use now plays in our public schools. Rather than using only standardized academic
assessments as the chief means of measuring students’ growth or regression, educators are now
expected to use a broader range of data – academic, social/emotional, and behavioral assessment
data - to inform instructional, programmatic, and intervention decisions. This DDM measures the
extent to which teacher or department teams utilize data that they already have, or may create, to
identify and address students’ learning needs, and subsequently make adjustments to instruction
and student interventions
The standards that informed this CO are listed in the Content Chart below and center on four
aspects of leaders’ support for teachers’ data use:
1.
2.
3.
4.
The extent to which educators use a range of student data at meetings
The extent to which educators engage in the analysis of student data at meetings
The extent to which educators use data at meetings to identify student strengths and needs
The extent to which educators use data at meetings to determine appropriate interventions
and adjustments to instructional practice
Assabet Valley Collaborative – DDM – Teacher Teams Use of Data 4
Content (Job Responsibility)
Weight
CO: Educators use a range of student data at meetings
Leadership Standard I-C: Assessment
 1-C-1 Supports educator teams to use a variety of formal and informal
methods and assessments, including common interim assessments that are
aligned across grade levels and subject areas.
Team Meeting Data Use Survey Items #5 and #6
CO: Educators engage in an analysis of student data at meetings
Leadership Standard I-C: Assessment
 I-C-2 Provides planning time and effective support for teams to review
assessment data and identify appropriate interventions and adjustments to
practice. Monitors educators’ efforts and successes in this area.
Team Meeting Data Use Survey Items #4, #7 and #8
CO: Educators use data at meetings to identify students’ strengths and needs.
Leadership Standard I-E: Data-Informed Decision Making
 1-E-3 Uses multiple data sources to evaluate educator and school
performance. Provides educator teams with disaggregated assessment data
and assists faculty in identifying students who need additional support.
Team Meeting Data Use Survey Items #9, #10
CO: Educators use data at meetings to determine appropriate interventions and
adjustments to instructional practice.
Leadership Standard I-C: Assessment
 1-C-2 Provides planning time and effective support for teams to review
assessment data and identify appropriate interventions and adjustments to
practice. Monitors educators’ efforts and successes in this area.
25% of the
measure
25% of the
measure
25% of the
measure
25% of the
measure
Team Meeting Data Use Survey Items #9, #10, #11
100%
Instrument
This measure consists of a Team Meeting Data Use Survey that serves to gather meeting
participants’ perceptions about their data use during two common planning meetings per month
throughout the school year. The survey requires less than five minutes to complete and consists of
14 items. These include four team meeting context questions – e.g., date, grade level, department
subject, name of person completing the survey. It also included ten core questions about team data
use, such as whether the team used data during their meeting and if so, what types of data,
followed by several questions about the purpose of their data use and actions the team decided to
Assabet Valley Collaborative – DDM – Teacher Teams Use of Data 5
take based on their data discussion. This survey tool, designed as a Google Form to facilitate
ongoing compilation of results, provides the only data that is formally tracked and evaluated for this
DDM. Although teams are asked to reflect on their data use by completing the survey as a group
during the final minutes of their meetings, they also have the option, if needed due to lack of time, to
ask the designated team leader or department head or a designated teacher on the team to
complete the survey within 24 hours of the meeting on the team’s behalf. Results of the survey are
evaluated quarterly using the School-Wide Team Data Use Rubric. When survey results are
downloaded as an Excel file, administrators may also disaggregate data to strengthen the
understanding of team-level patterns and progress over time, thereby informing more targeted
administrative supports.
NOTE that the school administrator will need to create four copies of the online survey 2, each titled
to represent the given period of the year. At the start of each new period (the beginning of
November, February, and April), send the new survey link to team leaders with a note to delete the
previous link. This allows a new data collection cycle by returning the Responses to zero; otherwise,
using the same Google Form will simply accumulate survey responses throughout the year, making
it significantly more difficult to complete the School-Wide Data Use Rubric at the end of each period
and to compare progress over time.
An optional Teacher Post-Observation Debrief Form is also provided in this DDM, as well. Although
not formally scored as part of the DDM, this tool aligns to the questions in the Team Meeting Data
Use Survey. It also is designed to assist administrators in aligning their classroom observation
debrief discussions with their efforts to support teachers’ effective data use during their team
meetings.
Administration Protocol
The Administration Protocol addresses how the measure is intended to be implemented to best
support a common conversation about growth, as well as to strengthen the consistency of
administration across schools in a district.
When is the measure administered?
This is a repeated measure where teachers’ common planning teams, presumed to meet at least
twice per month, are asked to complete an online version of the Team Data Use Survey at the close
of two meetings per month (and no later than 24 hours after any given team meeting) from the start
of school to the end of May.
The request to use the final five minutes of team meetings for this task is to convey respect for
teachers’ time and to avoid asking teachers to fit another task into their busy work days. It is
recommended that teams establish a predictable routine, such as completing the survey together at
the close of the first two meetings of every month. This will help to prevent teams from accidentally
2
If using Google Forms, go into the Team Meeting Data Use Survey Google Form and select “Make a Copy” from the
drop down “File” menu options.
Assabet Valley Collaborative – DDM – Teacher Teams Use of Data 6
completing the form more or less frequently, which could skew results. (See section on deviations to
the protocol, below.)
The administrator analyzes the survey results four times per year: (1) end of October, (2) end of
January, (3) end of March, and (4) end of May. Districts that have a different team meeting schedule
may need to modify the frequency that the survey is completed and/or compiled and analyzed.
Asking teams to complete the survey at two meetings per month helps to build in regular team
reflections regarding their data use and quarterly analysis by the administrator provides ongoing
information that is likely to prompt important adjustments to various team supports. This approach
ensures more effective and widespread data use as the school year progresses.
How is the measure administered?
Introducing the Team Data Use Survey to Staff
Prior to the first survey use, the administrator should facilitate collaborative meetings with teachers
to discuss the focus of this DDM. In particular, it is important to clearly communicate that the survey
is a DDM for the administrator (not the teachers). Teachers will be familiar with DDMs, but may not
realize that the administrator also has DDMs and is expected to show growth in leadership skills.
Also, the administrator should make clear that the DDM is aimed at school-wide progress with team
data use. This may mean the administrator sometimes needs to disaggregate the data to better
understand where or how to provide supports to teams most effectively, but the DDM results are
interpreted based on school-wide improvement trends. The administrator should work closely with
team leaders and department heads to explain and clarify the process, tools, and expectations prior
to the initial implementation of the DDM.
During September faculty meetings, the administrator should facilitate a comprehensive initial
discussion and/or professional learning on the role of data-based inquiry. The session will identify
student learning strengths and needs and, subsequently, determining appropriate interventions and
instructional adjustments to ensure all students progress in their learning. The administrator may
want to engage teachers in a text-based protocol to read and reflect on an article about
collaborative data use for student learning, ask teachers to work together to chart how their
teamwork might look different with more effective data use, or to generate a list of the additional
knowledge, skills, or resources they might need to use data more effectively. In particular, the
administrator will need to introduce the priorities for team data use with particular attention to
previous school or district expectations for the use of student learning data and the use of common
planning time. Teachers may express concerns or frustrations if they receive mixed messages from
the district, school, and/or union about the intended use of their common planning time.
During the September faculty meetings, the administrator must also introduce teachers to the Team
Data Use Survey and state explicitly that the administrator’s interest is in supporting teams to use
data effectively to achieve school-wide improvement. Administrators should point out the following:
o
Item #4 asks whether data was discussed and, although not all team meetings will
necessarily involve data use, the school is working to have more evidence-based team
meeting discussions about how best to advance students’ learning.
Assabet Valley Collaborative – DDM – Teacher Teams Use of Data 7
o
Items #5 and #6 relate to the range of data from which teams may want to draw when
investigating student learning needs.
o
Items #7 and #8 relate to the varied ways that team may discuss data, with some
discussions achieving greater depth and analysis than others. In particular, teams are
encouraged to drill down to the small- or sub-group and individual student levels in their
data discussions as this is more likely to inform classroom instructional decisions than
staying at the department, grade, or school levels.
o
Item #9 relates to the purpose of the team’s data use, with the aim of focusing on
students’ strengths, needs, and performance gaps, and identifying supports,
interventions, and instructional adjustments that will ensure the progress of all students.
o
Items #10 and #11 - both open-response items – ask teams to articulate very specifically
what teams are learning about students and their instructional supports as a result of their
data-based discussions and how they are going to apply what they have learned toward
next steps for students.
The administrator should acknowledge that the survey measures only the team’s perceptions in
these areas. Administrators should also note their intentions to attend and observe team meetings
across the school to better understand the survey feedback, to learn more about how teachers
actually work together in their teams, and to be able to identify and facilitate sharing of helpful team
practices across the school.
Throughout the year, the administrator must ensure and protect opportunities for regularly
scheduled common planning time and provide professional learning opportunities to support data
use for instruction and student learning. In between team meetings, continued conversation should
occur between the administrator and teachers to support teams’ analysis of student data. This may
take place during post-observation debrief discussions between administrators and individual
teachers, when administrators attend team meetings to observe team data use in action, or by
teacher or team request. Administrators should use information gathered from these interactions to
inform professional development planning.
Throughout these interactions, the administrator must engender teachers’ trust that the results of
the Team Meeting Data Use Survey will never be used in any punitive way, and that the
administrator is trying to use the results to improve his or her own practice to support teams
appropriately. Teachers may initially be concerned about what happens if their meetings do not
address the stated priorities or may express concerns that other factors, such as school or district
agendas, field trips, parent issues, or testing schedules, determine the extent to which they can
focus their team time on data use for students and instruction. The administrator should be ready to
think with teachers about how they, collectively, might shift current practices, such as the allocation
of time and tasks, to achieve progress. The administrator must then follow through on the results of
these conversations, both to earn teachers’ trust and to demonstrate growth with this DDM.
Assabet Valley Collaborative – DDM – Teacher Teams Use of Data 8
Email Communication
After the initial staff meeting discussions, the administrator must send an email to staff that clearly
and succinctly reiterates the purpose of the Team Meeting Data Use Survey, as noted above, and
requesting that teams collectively complete the survey during the final five minutes of at least two
team meetings per month. The administrator should also encourage teachers to let him or her know
if they have any questions or concerns about this request.
This email provides the link to the Team Data Use Survey, requests that team leaders bookmark the
survey site on the Internet browsers for use throughout the year, and includes the following
directions:
1. During the last five minutes of two team meetings per month, please engage your full team in
discussing and completing this brief 14-item survey. The purpose of the survey is to provide
me with information about the extent to which teams across the school are engaging in databased discussions about instruction and student learning and then determining how best to
take action to ensure that all students are progressing in their learning.
2. Periodically, I will analyze all survey responses to identify school-wide patterns and trends
over time with the aim of providing appropriate and helpful supports and professional learning
opportunities to ensure all teams are getting the most from their team meetings.
3. The aim is to strengthen our ability to use data to inform decisions about instruction and
student learning across the school. This means that data use in our common planning time
meetings leads to the identification of actions the team will take to strengthen instruction or
provide interventions or supports for student learning.
Ongoing Communications
Administrators should maintain the visibility of these common planning team priorities throughout
the year by providing feedback and follow-up to the staff about progress, sharing useful tools and
strategies from teams across the school, and/or providing professional learning opportunities and
new resources as part of ongoing staff meetings or development time. This improvement orientation
also reinforces the non-punitive nature of this work and the administrator’s interest in supporting
continuous improvement.
How are deviations to protocols addressed?
The administrator should periodically review the results of the Team Meeting Data Use Survey to
ensure that all teams have established a reliable routine for completing the survey twice per month.
Email reminders and/or one-to-one conversations with team leaders may be necessary to ensure
response rates remain high throughout the school year.
If online administration of the Team Data Use Survey is not an option, the administrator will need to
arrange for support staff to manage data entry based on return of paper versions of the survey.
Districts will need to modify the frequency of survey administration and data analysis of results
according to team meeting schedules and expectations in their local context.
Assabet Valley Collaborative – DDM – Teacher Teams Use of Data 9
It is possible that teams may complete the Team Meeting Data Use Survey more than two times per
month, either because they forgot if they had already completed it twice that month, or because they
want to provide more frequent updates to the school administrator. If this occurs, results may be
skewed by the over- or under-representation of particular teams. For example, if one team
frequently lacks a focus on data use, but completes the survey after every meeting – perhaps four
times a month instead of two – this might skew school-wide results toward looking like teams are
not making progress.
To avoid this situation, the administrator is encouraged to download survey results as an Excel file
and sort the data by team-level to gain a view of whether teams are, in fact, completing the Team
Feedback Survey twice per month. If not, the administrator should connect with particular common
planning teams to learn why surveys are not being completed, as requested, and to ensure that
staff understand that the results are to help the administrator improve supports and guidance to
teams and are not an accountability measure for teams or an assessment of individuals on the
teams.
To increase teachers’ commitment to these data-use priorities, the administrator may want to
involve teachers in further defining these criteria or survey items. It is important, however, to be
aware that changes may require revisions to other aspects of the DDM, such as the Core Course
Objective, survey items, and Assessment Blueprint. Teacher commitment and involvement are
strongly encouraged, but administrators must be aware of the subsequent changes that would need
to be made to the broader DDM and have the knowledge, skills, and/or support to make these
changes.
How will special accommodations be provided?
No special accommodations are needed for this indirect measure, particularly since teacher teams
are encouraged to collectively complete the Team Meeting Data Use Survey, so support is provided
within the team. However, administrators should work with teachers to understand the extent to
which data are being used in teams to serve the full range of students in the school. In addition,
administrators should provide supports and professional development to help teams understand the
kinds of modifications and accommodations that may be necessary to ensure that all students are
able to demonstrate the full extent of their learning through classroom assessments, thereby
strengthening the quality of the data the teams will use to determine interventions and instructional
adjustments.
Scoring Guide
To score the results of the Team Meeting Data Use Survey at the designated times (end of October,
end of January, end of March, and end of May), the administrator downloads the survey results for
that period (preferably as an Excel sheet) and analyzes results in relation to the criteria and
performance levels described on the School-Wide Team Data Use Rubric. When using Google
Assabet Valley Collaborative – DDM – Teacher Teams Use of Data 10
Forms, the percentages described in the rubric can be quickly collected from the provided Summary
Data sheet.
Note first that several survey items are included in the survey only to provide background
information, such as the date, the grade-level or department, and the role of the person completing
the form. Responses to these items are not evaluated in the School-Wide Team Data Use Rubric,
but do allow the administrator to gauge whether all teams are completing surveys twice per month
and whether any follow-up with team leaders is needed to maintain a high response rate.
In addition, there are two survey items that are not evaluated, but are included to provide
information to help the administrator determine appropriate team meeting supports. These items
include:
 Item 12: If you did not analyze data in your team meeting today, which data (if any) might have
been helpful to collect and/or review to inform your instructional planning and intervention
decisions?
 Item 13: What would help you and your team use data more effectively to inform instructional
decisions moving forward?
Also, note that the administrator’s analysis of these quarterly survey results occurs at the school
level and does not involve disaggregating the data to the team level. Administrators may pursue
team-level analysis to inform their team supports and planning for professional development
throughout the year. Scoring the DDM, however, only requires analysis of school-wide patterns and
trends over time.
The School-Wide Data Use Rubric is used to evaluate survey results. The administrator reads
across one row of the rubric at a time to determine the level of performance demonstrated across
the school for the given period of the year. Several rows in the rubric require summary statistics
from the Team Meeting Data Use Survey. If using Google Forms, the administrator enters edit
mode of the Team Meeting Data Use Survey Google Form and goes to the “Responses” menu
option – notice that Google tells how many survey responses have been recorded in parentheses –
then selects “Summary” from the “Responses” drop-down menu.
These “Summary” statistics provide the school-wide percentages needed to complete the SchoolWide Data Use Rubric.
In addition, the administrator must also carefully review and analyze the school-wide results for the
two open-response items to determine the appropriate level of performance on the rubric. If the
administrator finds that teams are not clear enough or are not elaborating enough in their responses
to provide useful information, the administrator should provide feedback to the team during the next
team meeting.
To illustrate, Item 10 – “Based on your discussion about data, what did your team learn?” – and
Item 11 – “What are the team’s next steps as a result of the data discussion today?” – will likely
Assabet Valley Collaborative – DDM – Teacher Teams Use of Data 11
elicit quite varied narrative responses from team to team or across survey periods. According to the
School-Wide Data Use Rubric, responses to Item 10 are evaluated in relation to the strand, Data
Use for Identifying Students’ Needs to see to what extent teams tended to specify team learning
about student needs or gaps in learning. The administrator then evaluates responses to Item 10
again in relation to the next row in the rubric: Determination of Interventions and Instructional
Adjustments. In this case, the same open responses for Item 10 need to be coded or highlighted
again, but this time for the extent to which teams have committed to actions related to intervention
strategies or instructional adjustments. Note: This is the only case where a single open response
item is evaluated twice for different criteria.
Although the open response items are more time-consuming to analyze, these qualitative
responses will likely provide more nuanced information about team data use. They will therefore
better inform the administrator’s subsequent discussions with teams and allocation of team supports
and resources.
Each performance level on the rubric is valued at a certain number of points, ranging from 0 points
(Emerging Team Data Use for Student Learning) to 25 points (Extending Team Data Use for
Student Learning) and including several levels in between. Note that there are also three levels
included without descriptors – shown as narrow columns marked with an asterisk. These columns
are for use when results for a single row cross two contiguous levels on the rubric. For example,
evidence for Data Use and Analysis might cross both the Developing and Solidifying performance
levels. The administrator can then assign points according to the level situated in between the two
levels; in this case, the in between is worth 12 points.
The administrator should maintain a tracking sheet, such as the following, to record results from the
School-Wide Data Use Rubric for each period during the year. The Administer may want to set up
more detailed records to track results at the team level, or at more frequent intervals to inform
thinking about team supports, resources, and/or interventions for progress over time, as well as the
results of those supports.
End of October
SCORING CHART
School-Wide Responses
End of January End of March
End of May
# Responses:
# Responses:
# Responses:
# Responses:
Rubric Points
Rubric Points
Rubric Points
Rubric Points
Points
Gained
Range of Data Sources
Items 5 & 6
Data Use & Analysis
Items 6 & 7
Identification of Student Strengths & Needs
Items 9 & 10
Determination of Interventions & Instructional Adjustments
Items 9 & 11
Total Points (from 100 possible pts)
Total Pts
Assabet Valley Collaborative – DDM – Teacher Teams Use of Data 12
Going down the column for the current time period, the administrator should insert into each white
cell the total rubric points for each of the four portions of the School-Wide Data Use Rubric. This
tracking will help the administrator see in which areas teams may need the greatest support. An
analysis of the open-ended response items (#10 and #11) should provide the administrator with
additional information about how best to target supports, resources, and/or professional learning.
In the far right column, administrators should calculate the gain in points for each rubric section
across the four periods shown on the chart in order to gain a view of particular areas of strength and
weakness at year’s end. This is done by subtracting the initial points recorded at the end of October,
from the final points recorded at the end of May. The total points gained during the year is recorded
in the far, lower-right cell in the chart.
A sample completed chart for the year might look like this:
End of October
SCORING CHART
School-Wide Responses
End of January End of March
End of May
# Responses:
# Responses:
# Responses:
# Responses:
Rubric Points
Rubric Points
Rubric Points
Rubric Points
Range of Data Sources
Items 5 & 6
0
8
Data Use & Analysis
Items 6 & 7
4
8
Identification of Student Strengths & Needs
Items 9 & 10
8
12
Determination of Interventions & Instructional Adjustments
Items 9 & 11
4
12
Total Points (from 100 possible pts)
Total Pts
16
40
Points
Gained
8
16
16
16
12
8
16
20
12
8
16
12
48
64
48
It is recommended that administrators collaborate with role-alike colleagues in the district to
calibrate their scoring practices with both the quantitative and qualitative data collected via the
Team Data Use Survey. Calibration strengthens the likelihood that scoring will be conducted in a
consistent and reliable manner over time and across schools. A sample calibration protocol can be
found at the Rhode Island Department of Education website:
http://www.ride.ri.gov/Portals/0/Uploads/Documents/Teachers-and-Administrators-ExcellentEducators/Educator-Evaluation/OnlineModules/Calibration_Protocol_for_Scoring_Student_Work.pdf.
This protocol can be modified for use with this DDM in a couple of ways. First, a calibration exercise
can be completed by administrators comparing their quantitative team data to discuss whether they
have organized and calculated their results, then applied these to performance level determinations
on the rubric, in the same ways. Second, administrators can code or highlight – then determine
rubric performance levels – for several common sets of sample open response data to gauge
whether they are interpreting these qualitative data in the same ways. Calibration is particularly
Assabet Valley Collaborative – DDM – Teacher Teams Use of Data 13
important for tuning administrators’ interpretation of qualitative data, as is required with items #10
and #11 in this measure.
Measuring Growth and Settings Parameters
To determine growth in team data use over the course of the year, the administrator subtracts the
total school-wide rubric score for the first period of the year (September through the end of October)
from the total rubric score for the final period of the year (beginning of April thorough the end of
May). For example, as shown in the sample score chart above, if one administrator’s school-wide
rubric score at the end of October was 16 points and at the end of May, the score was 64 points,
this represents a gain of 48 points.
The following Growth Parameters describe estimated growth bands for school-wide team data use,
representing estimations for low, moderate, and high growth. Parameters are based on systematic
estimations of results for a hypothetical school imagined by the developers of this DDM. Returning
to the example above, where the sample school gained 48 points during the year, this would be
evaluated as moderate growth, according to the Growth Parameters shown below.
Low Growth
<30 pt gain
Moderate Growth
30-50 pt gain
High Growth
>50 pt gain
Districts that aim to adopt or modify this DDM should revisit and refine these estimates based on a
careful review of data collected during the first years of administration in their own school sites. The
activity proceeds until historical trends can be established and used to inform and refine these
preliminary parameters.
Piloting
This DDM will be piloted with a subset of teacher teams in 2015-16. In addition to piloting the Team
Meeting Data Use Survey, administrators will be piloting the administration protocol, assessing the
appropriateness of the estimated growth parameters, and gauging the overall school climate for
implementing this type of DDM. The developers acknowledge that staff trust is a necessary precondition for the success of this DDM, as well as central office support for the priorities described in
this measure.
Assessment Blueprint
The assessment blueprint is not a task to be completed as part of the DDM, but is an elaboration of
the content table included in the introduction. It serves two purposes: (1) it is a roadmap for the
assessment development team to ensure balanced coverage of the most important content and (2)
it is a key for other potential users of the assessment by concisely indicating what content the
assessment is designed to measure, whether the goal is growth or target attainment, and the
Assabet Valley Collaborative – DDM – Teacher Teams Use of Data 14
difficulty of the items associated with each piece of content. (See pages 12 and 29 of Technical
Guide A for more information.)
Assabet Valley Collaborative – DDM – Teacher Teams Use of Data 15
Team Meeting Data Use Survey
Designed Using Google Forms and Utilizing Branched Questions
Assabet Valley Collaborative – DDM – Teacher Teams Use of Data 16
Assabet Valley Collaborative – DDM – Teacher Teams Use of Data 17
Assabet Valley Collaborative – DDM – Teacher Teams Use of Data 18
Assabet Valley Collaborative – DDM – Teacher Teams Use of Data 19
Assabet Valley Collaborative – DDM – Teacher Teams Use of Data 20
Assabet Valley Collaborative – DDM – Teacher Teams Use of Data 21
Assabet Valley Collaborative – DDM – Teacher Teams Use of Data 22
Assabet Valley Collaborative – DDM – Teacher Teams Use of Data 23
Date:
End of October
End of January
End of March
End of May
School-Wide Team Data Use Rubric
For each criterion (row), highlight the cell that most closely describes the demonstrated school-wide performance level
Criteria
Range of Data
Use
Items #5, #6
Data Use &
Analysis
Items #4, 7, 8
Emerging Team Data
Use for Student Learning
0 pts
Items 5 & 6: Overall,
teams report consulting a
small or limited set of
data sources
*
4
Performance Levels
Developing Team Data
Solidifying Team Data
Use for Student Learning
*
Use for Student Learning
8 pts pts
12
16 pts
Items 5 & 6: Overall,
Items 5 & 6: Overall,
teams report consulting
teams report consulting
some varied types of data
clearly varied types of
sources
data sources
Item 4: Teams report data
use occurring in less than
25% of all meetings
reported for this period
Item 4: Teams report data
use occurring in 26-50% of
all meetings reported for
this period
Item 4: Teams report data
use occurring in 51-70%
of all meetings reported
for this period
Item 7:Teams report
SHARING and
distributing data for more
than 40% of meetings
Item 7: Teams report
REVIEWING data –
sharing and discussing
data at a general or
introductory level for more
than 40% of meetings
and/or
Teams report less than
40% of the meetings
demonstrating in any
response category
Item 7: Teams report
ANALYZING data –
sharing and analyzing
data by sorting,
disaggregating, or
comparing data
and/or
ANALYZING and
CONCLUDING – includes
drawing clear conclusions
from the analysis of data
for more than 40% of
meetings
*
20
Extending Team Data
Use for Student Learning
25 pts
Items 5 & 6: Overall, teams
report consulting clearly
varied “academic” data,
clearly varied “other” types
of data, and additional data
sources (in “other” space)
Item 4: Teams report data
use occurring in more than
70% of all meetings
reported for this period
Item 7: Teams report
FOLLOWING UP with data
– analyzing data to assess
progress based on
decisions made at prior
meetings for more than
40% of meetings
Item 8: Teams report
Item 8: Teams report
Item 8: Teams report
Item 8: Teams report
analyzing data at the
analyzing data at the small
analyzing data at the
analyzing data at the small
small group or individual
group or individual levels
small group or individual
group or individual levels for
levels for less than 20%
for 20-33% of meetings
levels for 34-50% of
51-100% of meetings
of meetings
meetings
* Columns marked with asterisks indicate intermediate levels for use when evidence within a single row crosses varied performance levels. For example, evidence for Data
Use and Analysis might cross both the Developing and Solidifying performance levels, in which case the administrator would select the intermediate performance level, worth
12 points.
Assabet Valley Collaborative – DDM – Teacher Teams Use of Data 24
Criteria
Data use for
Identifying
Student Needs
Items #9, #10
Emerging Team Data
Use for Student Learning
0 pts
Item 9: Teams report
analyzing data to
determine student needs
or gaps in learning in
0-25% of meetings
Item 10: Majority of open
responses indicate team
learning that does not
mention student needs or
gaps in learning
Data Use for
Determining
Interventions
& Instructional
Adjustments
Items #9, #10,
#11
Item 9: Teams report
analyzing data to
determine appropriate
student support strategies
interventions, or
adjustments to instruction
in 0-25% of all meetings
reported for this period
Item 10: Majority of open
responses indicate team
learning that does not
mention student
interventions or
instructional adjustments
Item 11: Majority of open
responses do not mention
committing to intervention
strategies or instructional
adjustments
*
4
Performance Levels
Developing Team Data
Solidifying Team Data
Use for Student Learning
*
Use for Student Learning
8 pts pts
12
16 pts
Item 9: Teams report
Item 9: Teams report
analyzing data to
analyzing data to
determine student needs
determine student needs
and gaps in learning in
and gaps in learning in
26- 40% of meetings
41- 55% of meetings
Item 10: Majority of open
responses indicate team
learning about student
needs or gaps in learning
but are general, nonspecific, or basic
Item 9: Teams report
analyzing data to
determine appropriate
student support strategies
interventions, or
adjustments to instruction
in 26-40% of all meetings
reported for this period
Item 10: Majority of open
responses indicate team
learning about student
needs or gaps in learning
that are clear and specific
Item 10: Majority of open
responses indicate team
learning about student
interventions or
instructional adjustments,
but are general, nonspecific, or basic
Item 10: Majority of open
responses indicate team
learning about student
interventions or
instructional adjustments
that are clear and specific
Item 11: Majority of open
responses mentioning
committing to generallystated intervention
strategies or instructional
adjustments
Item 9: Teams report
analyzing data to
determine appropriate
student support strategies
interventions, or
adjustments to instruction
in 41-55% of all meetings
reported for this period
Item 11: Majority of open
responses indicate clear
and specific commitments
to intervention strategies
or instructional
adjustments
*
20
Extending Team Data
Use for Student Learning
25 pts
Item 9: Teams report
analyzing data to determine
student needs and gaps in
learning in 56-100% of
meetings
Item 10: Majority of open
responses indicate team
learning about student
needs or gaps in learning
that are clear, specific, and
cite evidence
Item 9: Teams report
analyzing data to determine
appropriate student support
strategies, interventions or
adjustments to instruction in
56-100% of all meetings
reported for this period
Item 10: Majority of open
responses indicate team
learning about student
interventions or instructional
adjustments that are clear,
specific, and cite evidence
Item 11: Majority of open
responses indicate specific,
measurable, achievable,
time-bound commitments to
intervention strategies or
instructional adjustments
TOTAL POINTS DEMONSTRATED
Assabet Valley Collaborative – DDM – Teacher Teams Use of Data 25
Teacher Post-Observation Debrief Form
Question numbers align to question numbers on the Team Data Use Survey Form
Teacher:
Evaluator:
Date and Time of Classroom Observation:
Focus of Observed Lesson:
Grade / Department:
Date and Time of Debrief Discussion:
1. Did you use any particular student data to plan today’s lesson?
2. & 3. Which type(s) of data did you use?
Academic Data (Check all that apply)
� Content-Specific Assessments (ex:
DIBELS, DRA, math fluency, STAR)
� Common Assessments
� Student Work
� MCAS / PARCC
� Unit Assessments
� Mid-term/final exams
� Formative Assessment (e.g., checks for
understanding, ticket to leave, “Do
Now,” daily quizzes)
� Performance Assessment
(demonstration of knowledge and skills)
� Capstone Project
� Student Portfolio
� Other
Other Student Data (Check all that apply)
� Student Support Team / Building-Based
Student Team notes or minutes
� Specialist Reports (ex: School
Psychologist reports)
� Free & Reduced Lunch data
� Teacher Observations (ex: Student
engagement, behavior)
� Guidance / School Adjustment
Counselor Notes
� Nurse Visits
� Absences
� Tardies
� Discipline Logs
� Bathroom Visits
� Detentions / Suspensions Reports
�Other
Assabet Valley Collaborative – DDM – Teacher Teams Use of Data 26
2. If you did not analyze data in preparation for today’s lesson, which data might have been helpful
to collect and/or review to inform your lesson plans and instructional decisions?
3. If you did work with data in preparation for today’s lesson, did you discuss or review these data
with your team or colleagues, or did you review the data individually?
How did you actually work with the data? Please describe (e.g., disaggregate, analysis over time).
4. What were you trying to understand by reviewing these data?
5. What was the purpose of your data use?
To determine students' strengths
To determine students' needs
To determine students' progress
To determine appropriate student support strategies or interventions
To determine adjustments we want to make to our instructional practice
To determine differences or gaps in students' learning
Other?
6. At what level did you focus your data analysis?
Individual student level
Small group level
Class level
Grade or department level
School level
Assabet Valley Collaborative – DDM – Teacher Teams Use of Data 27
7. What did you learn as a result of your data analysis?
What specific needs, trends, or patterns did you find?
How did these inform your lesson design or interventions?
8. Based on your lesson, what data did or will you collect next to determine whether your lesson
addressed the needs you identified?
9. What will you do next?
10. What would help you use data more effectively to inform interventions and instructional
decisions moving forward? How can I support you?
Consider: Does your team have the necessary skills to analyze and interpret student data? Does
everyone have access to necessary student data? Does your team need support linking data
analysis to next steps with instruction?
Assabet Valley Collaborative – DDM – Teacher Teams Use of Data 28