Running head: MULTI-TIERED SYSTEM OF SUPPORT INVESTIGATION
1
Determining the Impact of Implementing a Multi-tiered System of Support Structure on
Academic Achievement
Elizabeth F. Murray
East Carolina University
MULTI-TIERED SYSTEM OF SUPPORT INVESTIGATION
2
Problem of Practice
An affluent school district in southeastern North Carolina consistently posts
student achievement scores above the state average, averages of surrounding district, and
averages of comparably sized districts across the state on state end-of-year standardized
assessments. As Table 1 illustrates in regards to overall student achievement in grades
three through eight, the scores in Southeastern District (a fictitious name that will be used
subsequently to refer to the school district that is the focus of this study) rival those of the
largest public school district in the state (North Carolina Department of Public
Instruction, 2015).
Table 1
Overall Career and College Readiness Rates on State Standardized End-of-Year Tests
District
2010-2011
2011-2012
2012-2013
2013-2014
Comparably-sized
58.3%
59.7%
27.0%
39.6%
Largest
73.0%
74.2%
43.8%
56.9%
Southeastern
71.8%
72.9%
40.3%
54.0%
State
67.0%
67.5%
32.0%
45.8%
Surrounding
72.0%
69.7%
29.3%
44.4%
The exception to this praiseworthy performance in Southeastern District is the
difference in the achievement performance of black students when compared to their
peers in other districts and at the state level. Comparing end-of-year standardized reading
achievement test results of all students tested in grades three through eight to black
students tested, state level data show an average difference of just under 20 percentage
points while in this particular district the difference is more than 25 percentage points.
Table 2 outlines historical reading proficiency data for all students tested in third through
MULTI-TIERED SYSTEM OF SUPPORT INVESTIGATION
3
eighth grade compared to black students tested at the same grade levels at the state and
district level for four consecutive years (Public Schools of North Carolina, 2015).
Table 2
Reading Proficiency Rates on State Standardized End of Year Tests
2010-2011
2011-2012
2012-2013
2013-2014
State
70.7%
71.2%
43.9%
44.7%
Southeastern
75.0%
76.1%
50.9%
51.6%
State
54.2%
55.3%
25.6%
26.3%
Southeastern
49.3%
51.1%
21.6%
22.0%
All Students
Black Students
While the standard by which reading proficiency is determined has changed twice
in the past three years, the results have not. The data in Table 2 show that the
achievement gap in reading between all students tested and black students tested has
continued to widen in Southeastern District.
Within Southeastern District over past ten years the school district has shifted
from school attendance zone policies that tended to balance the schools both racially and
in terms of the percent of students considered economically disadvantaged to policies that
favor a neighborhood schools approach. A direct result of this decision by the board of
education is pockets of schools that are disproportionately filled with economically
disadvantaged, minority students that are centrally located in the downtown area of the
largest city in the district. These schools have experienced a decline in student
achievement as well as an increase in teacher and administrative turnover. Table 3
outlines representative schools of Southeastern District in terms of total enrollment,
percentage of black students enrolled and reading proficiency of each group.
MULTI-TIERED SYSTEM OF SUPPORT INVESTIGATION
4
Table 3
Enrollment, Demographics, Achievement, and Gap
Number
Tested
Percent
Black
Achievement
All Students
Achievement
Black Students
Gap
11,569
21.6%
51.6%
22.0%
29.6
Downtown Elem
156
84.6%
17.3%
15.9%
1.4
Largest Elem
359
16.4%
49.3%
35.6%
13.7
Similar Demographics Elem
166
23.5%
30.7%
12.8%
17.9
Highest Performing Elem
294
3.1%
79.3%
44.4%
34.9
Perimeter Elem
167
2.3%
74.3%
0%
74.3
Southeastern
Southeastern District has a rich history in its efforts to implement a model that
addresses increasing academic achievement and simultaneously decreasing the number of
students admitted into the special education program. Sparked by the reauthorization of
the Individuals with Disabilities Education Act (IDEA, 2004), approaches that entail the
use of data collection and early academic interventions prior to special education
identification began to evolve. These approaches were known collectively as Response
to Intervention (RtI) and were considered to be just one component of the larger problemsolving model (PSM). Canter (2004) defined the PSM as a cycle of identifying areas of
weakness, identifying and implementing evidence-based interventions, and collecting
data on student progress as a result of the intervention to assess the effectiveness of the
intervention. The premise of RtI is to provide academic, social, and emotional assistance
to all students as soon as they demonstrate a need rather than waiting until a student fails
and is at-risk of never catching up to peers (Buffum, Mattos, & Weber, 2010; Samuels,
2011; Sawchuk, 2011; Trotter, 2013).
MULTI-TIERED SYSTEM OF SUPPORT INVESTIGATION
5
As a result of the emphasis on the PSM, Southeastern District developed its own
process for identifying areas of academic weaknesses and providing and monitoring
interventions which they, too, called the Problem Solving Model. While the school
district referred to its process as PSM, it was essentially RtI. One distinction the district
saw as making its model more effective was that instead of using nationally normed
screening tools to identify areas of weakness, the district created its own universal
screening tools and normed the results at the district level. The reasoning for this was the
thinking that because students as a whole in this particular district were outperforming
students at the state and nation levels, the tools used to identify areas of weakness and the
corresponding norms for determining which students should be assessed further to
investigate the potential need for special education should reflect the local population.
Although a well-intentioned process, the PSM soon lost credibility and was even
challenged legally both because of the “home grown” probes and norms, and because the
process for identifying students potentially in need of special education services was
taking well over the mandated 90 day time limit. Daves and Walker (2012) discuss the
confusion between using RtI solely to identify specific learning disabilities (SLD) and the
requirements of IDEA stating, “To insist that RtI be used only for SLD identification
would significantly alter services rendered in a timely manner, as is required of school
districts” (p. 70). From the discredited PSM process, the district then moved to a more
state supported model of RtI using nationally recognized probes for identifying reading
weakness, but continued to norm the results locally still insisting that the norms should
reflect the data from the population locally as this was the student group that would be
impacted.
MULTI-TIERED SYSTEM OF SUPPORT INVESTIGATION
6
The most recent iteration of implementing an intervention model came on the
heels of the district’s hiring of a new director of special education which coincided with
the State’s most recent revision of the RtI process, a model known as the Multi-tiered
System of Support (MTSS). The State’s definition of MTSS is “a framework which
supports school improvement through engaging, research-based academic and behavioral
practices. NC MTSS employs a systems approach using data driven-problem solving to
maximize growth for all” (North Carolina Department of Public Instruction, 2014).
Other authors, (e.g. Berhardt & Hebert, 2011: Ehren, Lipson, & Wixson, 2013; O’Connor
& Freeman, 2012) who refer to RtI as a framework for continuous school improvement,
note the importance of district level leadership as a critical issue in implementing and
sustaining a process such as RtI. The district views the MTSS model as a general
education process. Southeastern District has one position dedicated to the
implementation of MTSS. The MTSS Coordinator position is currently housed in the
Instructional Services Division, but funded by special education dollars from the Student
Support Services Division.
A problem of practice is defined by the Carnegie Project on the Education
Doctorate (CPED) as “a persistent, contextualized, and specific issue embedded in the
work of a professional practitioner, the addressing of which has the potential to result in
improved understanding, experience, and outcomes” (CPED, 2014). The problem in
Southeastern district is that black students are not performing at acceptable rates of
academic achievement regardless of the demographic make-up and location of the
schools they attend. The impact of student failure in school is far-reaching. Students
who are not successful are at an increased risk of dropping out of school, poor health,
MULTI-TIERED SYSTEM OF SUPPORT INVESTIGATION
7
unemployment, poverty and incarceration (Buffum, Mattos, & Weber, 2010; Fuchs,
Fuchs, & Compton, 2012). With schools filled with children who are at significant risk
of academic failure, the academic achievement problem must be addressed.
Purpose
Southeastern District is in the first of a three year plan for implementation of the
MTSS model. District leadership believes that the MTSS model addresses the following
three academic improvement goals: “1) Decrease the gap in proficiency between the
lowest and highest performing schools, 2) Bridge the academic achievement gap
evidenced by data in Black/White student academic performance and 3) Increase
awareness of personalized learning systems that maximize student achievement including
data rich learning profiles, customized learning paths, and proficiency-based progress”
(NHCS District Implementation Plan, 2014). This project will evaluate the effectiveness
of Years 1 and 2 of the implementation of MTSS model in terms of closing the
achievement gap between its lowest and highest performing schools and the white/black
achievement gap. The study will focus primarily on Downtown Elementary School
which is the lowest performing school in Southeastern District, however there is no
black/white achievement gap due to the school’s population which is 85% minority.
Improvement Goal
The percentage of black students at DES in grades three through five scoring at
the standard for proficiency on end-of-year state reading tests declined from 21.2% in
2012-2013 to 15.9% in 2013-2014. Even with the implementation of a new 5 level
achievement system in 2013-2014 with levels three through five being considered
proficient only 31.8% of the black students were deemed proficient in reading. The
MULTI-TIERED SYSTEM OF SUPPORT INVESTIGATION
8
difference in these two measures for the 2013-2014 school year indicates that of the
31.8% of students considered to be proficient, half were only minimally proficient.
DES has set an improvement goal of increasing the reading proficiency measure
at levels four and five for black students by 16 percentage points each year over the next
two years. This would bring the reading proficiency percentage up to the expected
federal proficiency rate of 47.8% for black students by the end of the 2015-2016 school
year. Table 4 outlines the projected results needed to attain this goal. The projections are
based on current enrollment figures. There is no anticipated change in the school
attendance zone boundaries.
Table 4
Projected Achievement Data
Number Proficient
at Levels 4-5
Number Tested
Percent Proficient
at Levels 4-5
2013-2014
(actual)
2014-2015
(projected)
2015-2016
(projected)
21
47
80
132
149
168
15.9%
31.9%
47.9%
Questions and Tasks
The evaluation of the effectiveness of the MTTS implementation will seek to
answer the following questions by implementing the tasks associated with each question.
How do teachers define the core instructional program?
Beginning with the 1994 reauthorization of the Elementary and Secondary
Education Act (ESEA) and continuing with the No Child Left Behind (NCLB) legislation
in 2001, a standards driven approach to instruction in which all students, including most
students with disabilities, are taught and assessed on a uniform set of rigorous standards.
MULTI-TIERED SYSTEM OF SUPPORT INVESTIGATION
9
The results from these assessments are used to set state, district, and school level basis for
academic accountability. The standards-based approach is intended to decrease the
achievement gap between enfranchised groups and disenfranchised groups of students.
The rationale is that with the right general education strategies and support in place, the
high incidence of disabilities will decrease (Fuchs, Fuchs, & Stecker, 2010).
The question then becomes what is the right general education setting with the
necessary programs, supports, and strategies that meet the needs of the majority of the
population? Does this core instructional program vary from school to school? How do
teachers and school staff define the core instructional program?
To answer these questions I will conduct interviews with a representative group of
stakeholders from the district. This group will include the assistant superintendent of
instruction and academic accountability, the district MTSS coordinator, the school level
MTSS coordinator, the school instructional coach, the school administrators, and a group
of 5-8 teachers. The results will be shared with the interview participants and the MTSS
leadership team to raise awareness of possible inconsistencies in the understanding and
implementation of the core instructional program. The intention is that this formative
feedback will be used to adjust plans for the second year of implementation and lead the
MTSS team to strengthen its foundational understanding.
What universal screening tools are used? How does the use of local norms impact
the placement of students into tiers of support? How is fidelity of implementation
monitored?
One aspect of a tiered system of support is that students’ needs are identified early
through the use of reliable screening tools (Mellard, McKnight, & Jordan, 2010). As
MULTI-TIERED SYSTEM OF SUPPORT INVESTIGATION
10
stated earlier, in the evolution of MTSS the district moved away from “home-grown”
screeners to nationally normed universal screening tools but continues to use norms
developed by the school district to monitor student progress from the beginning of the
year (BOY), through the middle of the year (MOY), and to the end of the year (EOY).
An issue that arises from the use of local norms is the potential negative impact
on certain populations of students. It is already exemplified by historical test data that
students in this particular district outperform students at the state level. So too, data has
shown that black students in this district are achieving at a difference of just under 30
percentage points when compared with their peers in the district. Does the use of local
norms place the black students at DES at a disadvantage when trying to meet BOY,
MOY, and EOY benchmarks?
To examine the potential unintended consequences of using local norms to
monitor student progress, a list of universal screening tools will be compiled along with
the local norms benchmarks as well as the national norms benchmarks. Data collected
from the three benchmarking periods for DES will be analyzed to highlight differences in
tier placements based on local norms versus national norms. Findings will be presented
to district level stakeholders with recommendations for possible policy changes.
Why are
Study Plan
The study plan will span two years and evaluate Southeastern District’s ability to
address academic achievement issues concerning the black/white achievement gap and
the gap that exists between the lowest and highest achieving schools through the
successful implementation of the MTSS model. Years 1 and 2 of the implementation will
MULTI-TIERED SYSTEM OF SUPPORT INVESTIGATION
11
be evaluated. Formative findings and recommendations will be made periodically
throughout both years with summative findings and recommendations made after the
second year of implementation. The intention is that the summative findings and
recommendations will guide the third year of implementation.
Activities to be completed during year one include gathering baseline data,
attending MTSS professional development and implementation team meetings,
establishing a positive working relationship with stakeholders, and reviewing appropriate
state, district, and school level MTSS documents. Examples of baseline data include
state end-of-year assessment scores in reading, benchmark data collected from universal
screeners in reading, and demographic data. Throughout the first implementation year a
series of meetings and professional development for district and school level
implementation teams will be carried out. I will attend the meetings and professional
development activities as a means to gain understanding of the MTSS process and to
establish relationships with stakeholders. Also during year one implementation
documents created at the state, district, and school level will be examined. Documents
include the district implementation plan, readiness surveys, MTSS meeting protocols, and
monitoring tools.
Year two will include data analysis coupled with periodic formative feedback
evaluations. Data that might be examined include the percentage of students moving
between intervention tiers, numbers of students involved in specific research-based
interventions, and the fidelity with which MTSS meeting protocols are followed.
The study plan will continue to develop and evolve during the implementation
timeframe based on input from stakeholders.
MULTI-TIERED SYSTEM OF SUPPORT INVESTIGATION
12
References
Bernhardt, V. L., & Hebert, C. L. (2011). Response to intervention and continuous school
improvement: Using data, vision, and leadership to design, implement, and
evaluate a schoolwide prevention system. Kentucky: Routlegde.
Buffum, A., Mattos, M., & Weber, C. (2010). The why behind RTI. Interventions that
Work, 28(2), 10-16.
Canter, A. (2004). A problem-solving model for improving student achievement.
Principal Leadership: High School Edition, 5(4), 11-15.
Conaway, E. (2014). Assessing results of years one, two, and three implementation of
response to intervention. (D.Ed., University of Delaware). ProQuest Dissertations
and Theses, Retrieved from
http://search.proquest.com.jproxy.lib.ecu.edu/docview/1622927661?accountid=10
639.
Daves, D. P., & Walker, D. W. (2012). RTI: Court and case law--confusion by
design. Learning Disability Quarterly, 35(2), 68-71.
Ehren, B. J. (2013). Expanding pockets of excellence in RTI. Reading Teacher, 66(6),
449-453.
Fuchs, D., Fuchs, L. S., & Stecker, P. M. (2010). The "blurring" of special education in a
new continuum of general education placements and services. Exceptional
Children, 76(3), 301-323.
Mellard, D., McKnight, M., & Jordan, J. (2010). RTI tier structures and instructional
intensity. Learning Disabilities Research & Practice, 25(4), 217-225.
MULTI-TIERED SYSTEM OF SUPPORT INVESTIGATION
13
Moates, C. (2014) NHCS district implementation plan for multi-tiered system of support
3 year plan. Unpublished internal document.
North Carolina Department of Public Instruction. (2014). Multi-tiered system of support
factsheet. Retrieved from:
http://mtss.ncdpi.wikispaces.net/file/view/NC%20MTSS%20Factsheetpost.pdf/53
5849612/NC%20MTSS%20Factsheetpost.pdf.
North Carolina Department of Public Instruction. (2014). Reports of disaggregated state,
school system (LEA) and school performance data. Retrieved from:
http://www.ncpublicschools.org/accountability/reporting/leaperformancearchive.
O'Connor, E. P., & Freeman, E. W. (2012). District-level considerations in supporting
and sustaining RtI implementation. Psychology in the Schools, 49(3), 297-310.
One Hundred Third Congress of the United States of America. (1994). Improving
America’s school act of 1994. Retrieved from:
http://www2.ed.gov/legislation/ESEA/index.html.
Sawchuk, S. (2011). RTI makes few inroads into the nation's education schools.
Education Week, 30(22), 10-11.
Trotter, G. (2013, March 26). Subtle changes ahead for special education in District 36.
Chicago Tribune. Retrieved from http://articles.chicagotribune.com/2013-0326/news/ct-tl-ns-0328-winnetka-special-ed-20130327_1_special-educationschool-districts-rti.
U.S. Department of Education. (2004). Individuals with disabilities education act.
Retrieved from: http://idea.ed.gov/explore/home.
MULTI-TIERED SYSTEM OF SUPPORT INVESTIGATION
14
U.S. Department of Education. (2001). No child left behind act of 2001. Retrieved from:
https://www2.ed.gov/nclb/overview/intro/execsumm.pdf.