Teacher Impact on Student Proficiency and Growth
Catholic educators are called now more than ever to be able to demonstrate that students
are proficient, and achieve more than a year’s growth during a school year, because they
attend Catholic schools.
Ronald W. Costello, Barbara Shuey, Peggy Elson and Kathy Mears
Catholic schools are committed to providing strong academic environments, which enable
students to grow to their full potential. We must demonstrate that we are constantly making this
difference. Through more than 20 years of research, William Sanders found that classroom
teachers are the most important factor in achieving gains in student achievement. Reports by
education researchers John Kain, Eric Hanushek, William Sanders and others have demonstrated
that good instruction is 15 to 20 times more powerful than family background and income, race,
gender and other explanatory variables. To be successful, Catholic educators are called now
more than ever to be able to demonstrate that our students are proficient and achieve more than a
year’s growth during a school year.
Through a school improvement grant from Lilly Endowment Inc., the Archdiocese of
Indianapolis used the SAS-EVAAS (William Sanders) value-added analysis for the last four
years. The analysis determined whether the school and classroom teacher achieved a year’s
academic growth with the students they serve. This article will explain the value-added model
utilized to report teacher impact and discuss the findings on student achievement. In order to do
this, it is important to understand how both proficiency and growth influence student
achievement.
Possible School Outcomes
Chart 1
Theodore Hershberg,
Writing about “ValueAdded Assessment and
System Reform” in an
article in the Phi Delta
Kappan (2005), describes
the possible school and
teacher outcomes on
student achievement.
This effect is a balance
between two factors –
proficiency and growth.
(Chart 1) How the teacher
balances the two in the
classroom can determine
whether the students reach
their full potential.
Proficiency is the ability
Possible School Outcomes
P
r
o
f
i
c
i
e
n
c
y
High Proficiency
Low Growth
High Proficiency
High Growth
Low Proficiency
Low Growth
Low Proficiency
High Growth
Growth
Theodore Hershberg Value-Added Assessment and Systemic Reform, PDK, Dec. 2005
1
of students to meet an expected level of academic performance based upon standards. Growth is
the gain in student achievement from one year to the next.
Students in the Archdiocese of Indianapolis take the Indiana Statewide Test of Educational
Progress (ISTEP) in grades 3 through 10. The ISTEP was developed by the Indiana Department
of Education to report student progress in meeting the Indiana academic standards. It is the
primary measure of student growth and progress in the state. Because all of the archdiocesan
schools are accredited by the Indiana Department of Education, the archdiocese has been able to
report that our students are proficient in meeting the Indiana state standards. But this tells little
about the impact that the teacher has on student achievement from one year to the next. In order
to do this, the archdiocese needs to measure and report annual growth. Value-added is our model
for this purpose.
Why Value-Added?
It is vital to understand why a value-added analysis is informative. The first step is to be able to
look at student progress over time. Chart 2 shows the average number of students passing both
the English/language arts and mathematics ISTEP assessments at grades 3 through 10 for school
year 2006-2007. The three groups compared are archdiocese, nonpublic schools and state
average.
Chart 2
2006-07 Average Passing E/LA & Math
Why Value-Added?
100%
95%
90%
85%
90%
88%
88%
86%
93%
90%
89%
88%
92%
89%
91%
87%
82%
82%
80%
75%
70%
92%
91%
74%
73%
72%
74%
71%
68%
66%
65%
64%
60%
55%
50%
3rd
4th
5th
6th
7th
8th
9th
10th
At what grades is progress adequate?
Public School Average
Indiana Non-Public Schools Average
Archdiocese of Indianapolis Average
Indiana Proficiency
The chart displays a trajectory using the average number of students passing the
English/language arts and mathematics sections for the archdiocese (13,186 students), non-public
schools (30,526 students) and public schools (604,694 students). The present level of
proficiency on the ISTEP in Indiana is a 65 percent passing rate. This chart raises concern about
2
the lack of growth at the upper grades, yet this is not a value-added comparison because there is
no determination about expected level of growth from one year to the next. We could be seeing
from the chart the difference in student ability versus the impact that the school or teacher has
had on student achievement. In order to determine the impact of the school or teacher, we need
to match individual student achievement data from one year to the next and determine an
expected level of growth.
ISTEP Value-Added Results
For the last four years, the archdiocese has compared test results from year-to-year using the
value-added model developed by Sanders and now administered by the SAS®
EVAAS® Institute in North Carolina. The assumption of this model is that for a student to show
growth, the student must achieve a “gain score” that is greater than one-year’s growth.
For the first three years, the archdiocese used the Terra Nova test by CTB McGraw-Hill for this
comparison because ISTEP did not test students at all grades. During the 2005-06 school years,
with two years of ISTEP results from grades 3 through 10, we were able to get value-added
results for 29 schools using the ISTEP scores.
The data was collected from eight schools participating in the improvement efforts. An
additional 21 schools were part of the control group, which provided needed data to make the
value-added predictions for the eight schools. As Chart 3 shows, of the 29 schools, 13 schools
demonstrated a year’s growth, which is a three (3) or No Discernible Difference. Eight schools
had value-added growth and another eight had no value-added growth at a level to be considered
significant. This analysis assisted us in demonstrating both proficiency and growth for the 29
schools.
Chart 3
Archdiocese of Indianapolis
14
12
Number of Schools
Value Added
Score:
5 = +2
4 = +1
3 = NDD
2 = -1
1 = -2
10
8
6
4
2
0
Number of Schools
1
2
3
4
5
3
5
13
5
3
2005-06 ISTEP+ Value Added Score
3
Proficiency and Growth Results
For the 29 schools for which we had value-added scores, we created a chart to display
proficiency and growth (Chart 4). Based upon the five-point value-added score, each school was
placed into one of three categories. No growth was a one or two value-added score, average
growth was a three, and high growth was a four or five value-added score. Along with the valueadded score we used the combined average score in passing both English/language arts and
mathematics subject areas for the school. The student must pass both parts to be considered
passing, which is a higher expectation than the average passing used in the initial chart. For the
state of Indiana, the combined score is about 10 percentage points lower than the average score
at each grade level. The archdiocese used the combined average because we wanted higher
expectations for our schools than was required by the state of Indiana.
Chart 4
2005 ISTEP+ Proficiency & Growth
100%
15
11
7
90%
18
14
8
25
17
5
Proficiency
80%
1
4
70%
28
26
6
2
27
24
10
29
16
20 21
9
22
65% Proficiency
3
23
13
60%
12
50%
19
40%
30%
0
5
No Growth
10
15 Growth
Average
20
25Growth
High
30
Growth
In analyzing the results, seven of the 29 schools had both high proficiency and growth. Based
upon possible school outcomes in the Hershberg chart, we expect all schools to be proficient and
show growth. In order to accomplish this, the teacher needs to understand what is expected,
analyze the data and modify instruction in the school year in order to achieve both high
proficiency and growth with all students.
This year we will provide growth and proficiency reports for all 67 archdiocesan schools and not
just the pilot schools in our original improvement effort and we will release proficiency and
growth data to all archdiocesan schools so teachers can plan to modify instruction.
4
Conclusion
Battelle for Kids, a nonprofit organization in Columbus, Ohio, was established to “enhance
student learning by bringing clarity to school improvement.” This improvement in teaching and
learning is to be achieved through The Power of Two: Progress and Achievement. Progress is
defined as student growth and performance from one year to the next, and achievement is
meeting the expected level of student performance. If we use the terms from Hershberg,
progress is growth and achievement is proficiency; how Catholic schools address proficiency and
growth will determine our success in this era of teach accountability.
When we are able to show that our Catholic teachers are effective because of the high percentage
of students passing a standards-based assessment such as the ISTEP, and students in our
classroom are gaining a year’s growth in a school year, then we will have met our goals.
Providing teachers with information about student proficiency and growth, and measuring the
impact teachers have on student learning help identify best practices that can be replicated and
rewarded. The goal of all Catholic educators is to demonstrate that they have achieved growth
and proficiency for all students. This should convince all about the value of excellent teachers in
our Catholic schools.
Dr. Ronald W. Costello has been superintendent of Catholic education for the Archdiocese of Indianapolis for
the past five years. The archdiocese has 68 schools with almost 23,000 students in kindergarten through 12 th
grade. Prior to working in Catholic education, he worked for more than 30 years in public education as a
teacher, counselor and administrator at both the building and central office level (rcostello@archindy.org).
Barbara Shuey is director of Catholic Education Services for Teachscape, Inc. Prior to this position, she served
in the Archdiocese of Indianapolis for 25 years as elementary principal, high school president and, most recently,
as associate director of the Center for Catholic School Excellence in the Office of Catholic Education. She
worked for seven years as school administrator of the Indiana United Methodist Children’s Home. She currently
is a lecturer in education at Marian College in Indianapolis (bshuey@sbcglobal.net).
Peggy Elson is in her third year as TAP program director for the Archdiocese of Indianapolis. She taught for 25
years at Saint Matthew School followed by two years at Saint Lawrence School as TAP master teacher
(pelson@archindy.org)
Kathy Mears is associate director for school learning resources for the Archdiocese of Indianapolis. During her
25 years in Catholic education, she has served as a principal and as an elementary and high school special needs
teacher. She is a lecturer in education at Marian College in Indianapolis (kmears@archindy.org).
References
A Toolkit for School Leaders: Understanding & Using Value-Added Analysis Battelle, for Kids, 2006.
Costello, R., Shuey, B. & Elson, P. Performance pay − a case for corporate and foundation support. Momentum, 37,
34-37.
5
Graphical Summary of Educational Findings from the Tennessee Value-Added Assessment System (1996).
Knoxville, TN: University of Tennessee Value-Added Research and Assessment Center.
Hershberg, T., (December 2005). Value-added assessment and system reform.
Phi Delta Kappan.
Kain, J.F. (November 17, 1998). The impact of individual teachers and peers on individual student achievement.
Association for Public Analysis and Management.
Mahoney, J.W. (March/April 2006). How Value-Added Assessments Helps Improve Schools.
Phi Delta Kappa Edge.
Sanders, W. L. (2003). Beyond No Child Left Behind. Presented at the 2003 Annual Meeting American
Educational Research Association, Value-Added Approaches to School Accountability: Results and Lessons from
an Evaluation, Chicago, IL.
Sanders, W. & Rivers, J. (1996). The Cumulative and Residual Effects of Teachers on Future Student Academic
Achievement. University of Tennessee Value-added Research and Assessment Center.
Adapted from an article in Momentum, the Official Journal of the National Catholic Educational Association,
April/May 2007. Used with permission. Contact: Barbara Keebler, NCEA, (202)378-5762.
6