Abstract Title Page
Not included in page count.
Title:
An Analysis of Student Achievement Growth, Teacher Working Conditions and Qualifications,
and School Choice
Author(s):
Marisa Cannata, Ph.D., Vanderbilt University
Roberto V. Peñaloza, Vanderbilt University
2009 SREE Conference Abstract Template
Abstract Body
Limit 5 pages single spaced.
Background/context:
Description of prior research and/or its intellectual context and/or its policy context.
As the most important resource for student learning, the academic success of any school
depends on high quality teachers providing high quality instruction. Yet the impact of school
choice on teachers and their work environments has received less attention than other
components of school operations. Previous research points to observable differences in the
qualifications of teachers in charter, magnet, and private schools compared to their colleagues in
traditional public schools (Baker & Dickerson, 2006; Cannata, 2008). Additional work has
explored the pay, personnel practices, and professional community of charter schools (Cannata,
2007; Goldring & Cravens, 2008; Harris, 2006; Podgursky, 2008). Less research has focused on
the structure of teachers’ work inside various types of schools, their career decisions, and
professional development.
There are two key reasons for understanding both who teaches in various types of schools
and what their work lives are like. First, increasing school choice creates options for teachers as
well as students and the work environments created by different forms of schools can impact the
overall teacher labor market. Second, teacher qualifications, professional development, and the
structure of teaching inside schools have implications for the effectiveness of various school
types and their impact on student learning. Schools of choice may increase their effectiveness by
managing their human capital resources and organizing teachers’ work so that they can focus on
instructional matters. Indeed, preliminary work on charter management organizations suggests
that many charter management organizations use a human capital strategy to focus on student
achievement (National Charter School Research Project, 2007).
Purpose/objective/research question/focus of study:
Description of what the research focused on and why.
This paper explores the following question: How do variations in the work lives of
teachers across charter, magnet, private, and traditional schools contribute to differences in
student achievement across these school types? In doing so, this paper will focus on teacher
qualifications, the amount and type of professional development in which they engage, their
classroom contexts, the presence of barriers to their instruction, and teachers’ influence over
school decisions. We hypothesize that fewer instructional barriers, more effective professional
development, and greater influence over school decisions will be associated with higher student
achievement in schools of choice.
Utilizing data from teacher surveys, we examine the qualifications, professional
development, and working conditions of teachers across school types. Previous research has
highlighted the importance of teachers to student achievement, yet few teacher characteristics
have been consistently linked to greater effectiveness (Goe, 2007). Part of this difficulty in
establishing relationships between teacher characteristics and instructional effectiveness is due to
a reliance on administrative data with few details on the organization, context, and ongoing
2009 SREE Conference Abstract Template
1
professional development that shape teachers’ instruction. The rich information about teachers’
work lives available in these data allow a more detailed examination of the components that lead
to enhanced student outcomes. Further, a key argument for school choice is that giving greater
flexibility to schools in staffing practices may lead to improved school outcomes (Podgursky,
2008). Thus exploring the extent to which the staffing practices of choice schools contributes to
any observed differences in student achievement can help evaluate whether choice schools are
meeting this ideal.
Setting:
Specific description of where the research took place.
The data come from surveys of teachers in charter, magnet, private, and regular public
schools. The schools are located in urban, suburban, and rural contexts across 24 states. The
schools all participate in the Northwest Evaluation Association (NWEA) assessment program
and student achievement data in mathematics, reading, and language usage come from NWEA
assessments.
NWEA administers state-aligned, computerized adaptive assessments in both the fall and
spring of each academic year in reading, language usage, and mathematics. These assessments
reference a single, cross-grade, and equal-interval scale developed using Item Response Theory
methodology (Hambleton, 1989; Ingebo, 1997; Lord, 1980). The RIT scale is based on strong
measurement theory, and is designed to measure student growth in achievement over time.
NWEA research provides evidence that the scales have been extremely stable over twenty years
(Kingsbury, 2003; Northwest Evaluation Association, 2002, 2003).
Population/Participants/Subjects:
Description of participants in the study: who (or what) how many, key features (or characteristics).
The data come from a sample of 6,175 teachers in 283 schools, with a response rate of 74
percent. Surveys were collected from teachers in a sample of matched pairs of choice schools
and regular public schools. All charter, magnet, and private schools in the NWEA were invited to
participate in the study. Traditional public schools were matched to schools of choice based upon
grade range, racial-ethnic and socioeconomic composition, initial achievement scores, and
proximity. The final sample includes 115 charter schools, 33 magnet schools, 17 private schools,
and 118 traditional public schools. Within these 283 schools our analyses focus on a student
sample of over 20,000 students across all grade levels.
Student achievement data come from the NWEA assessments in reading, language usage,
and mathematics and also includes student demographic characteristics. The longitudinal nature
of the achievement data allow for analyzing both achievement status and growth.
2009 SREE Conference Abstract Template
2
Intervention/Program/Practice:
Specific description of the intervention, including what it was, how it was administered, and its duration.
Charter, magnet, and private schools and traditional public schools; teacher
qualifications, professional development, and work context as measured by a teacher survey
Research Design:
Description of research design (e.g., qualitative case study, quasi-experimental design, secondary analysis, analytic
essay, randomized field trial).
Statistical Survey, Quasi-experimental, Statistical Modeling
Data Collection and Analysis:
Description of plan for collecting and analyzing data, including description of data.
Analyzing the effects of school choice requires accounting for problems of self-selection
into choice schools (Hoxby & Murarka, 2008). Schools of choice may serve different types of
students as families and students can self-select into them. Although our data do not permit
analyses based on a randomized design (e.g., winners and losers of school lotteries), we have
large enough samples and a number of observable characteristics to implement quasiexperimental designs relying on propensity score matching. We matched choice schools with
regular public schools using several observables in administrative data before administering
surveys. In addition, we examine how the in-school conditions in the different school types
mediate the effects of school type on students’ achievement growth.
We estimate a series of three-level, hierarchical linear models to estimate achievement
growth nested within students, teachers, and schools (Raudenbush and Bryk, 2002; Singer and
Willett, 2003). At level 1, student achievement (whether reading, language usage, or
mathematics) is represented by an individual growth trajectory:
ΔAchievementikj=π0kj+ π 1kj(Student Characteristics)ikj+ ε ikj
where ΔAchievementikj is the observed achievement gains per month of student i between the fall
and spring administrations in teacher k in school j (i.e., achievement score growth normalized
over the time between test administrations); π0kj is the mean per month gain in achievement
across students within teacher k and school j, conditioned on student background characteristics
(i.e., race-ethnicity, gender, free/reduced lunch status, and grade level). We will assume a simple
error structure for εikj, i.e., that it is independently and normally distributed with a mean of 0 and
constant variance.
We assume that the average achievement gains vary across teachers, represented by the
following level-2 equations:
π0j =β00kj+β01kj(Teacher Demographics) + β02kj(Teacher qualifications and assignment)ikj
+ β03kj(Teacher working conditions)ikj + β04kj(Professional development)ikj + r0ikj
2009 SREE Conference Abstract Template
3
where β00kj represents the mean achievement gain per month in teachers or classrooms within
school j. Teachers’ demographics include race-ethnicity and gender. Teachers’ characteristics
and assignment include years of experience, highest degree earned, experience working in
another field prior to teaching, selectivity of undergraduate college, part-time assignment,
permanent substitute teacher, and participation in TFA or similar program. Teacher working
conditions include number of students taught, percentage of students with special needs,
presence of resource shortages (alpha=.89), disengaged students (alpha=.86), teacher climate
(alpha=.72), perception of student workload (alpha=.77), and teacher influence over school
decisions (alpha=.85). Professional development measures include whether the professional
development involves collective participation (alpha=.78), active learning (alpha=.88), reformoriented activities (alpha=.76), coherent (alpha=.87), and total hours of professional development
in the subject assessed.
In addition, we assume that the teacher or classroom gains in achievement vary across
schools, represented by the following level-3 equations:
β00kj =γ000+ γ001(School Type)kj + γ002(Other school-level characteristics)kj + u00j,
where γ000 represents the mean achievement gain per month across schools. School type consists
of dummy measures for charter, magnet, private compared with the matched traditional public
schools. Other school characteristics include school demographics and other measures of school
context.
Findings/Results:
Description of main findings with specific details.
Preliminary results indicate that the characteristics, qualifications, and work contexts of
teachers in charter and private schools have some differences from traditional public schools,
while magnet school teachers and their contexts are more similar to traditional public schools.
Charter school teachers have, on average, about half the years of experience as teachers in
magnet, private, and traditional public schools and charter and private school teachers are less
likely to have advanced degrees. Private school teachers are also less likely to have entered
teaching from another career and more likely to have part-time appointments.
Private school teachers appeared to have better working conditions as they reported fewer
resource shortages, perceived the teacher climate and student disengagement was less of a barrier
to their instruction, greater influence over school decision-making, and—with charter school
teachers—reported instructing fewer students. Private and magnet school teachers perceived
their student workload and students’ needs to be less of a barrier to their instruction.
While teachers across all types of schools reported similar number of hours of
professional development focused on math and reading/language arts instruction, the reported
differing characteristics of this professional development. Charter school teachers reported more
professional development in which they participated with their colleagues and more reformoriented professional development. Private school teachers participate in professional
development that provides fewer opportunities for active learning and learning reform-oriented
instruction. Given previous research on features of professional development that are associated
with greater student learning (Garet et al, 2001), it would appear that charter school teachers
2009 SREE Conference Abstract Template
4
participate in somewhat more effective professional development and private school teachers
participate in less effective professional development.
Despite these observed differences in the qualifications, professional development, and
work contexts of teachers across school types, few of these differences were related to student
achievement gains. Compared to traditional public schools in models with grade level dummies
only, achievement gains per month were larger in charter schools, with no differences for private
or magnet schools. Consistent with prior research on teacher qualifications, few qualifications
had a significant effect on student achievement gains. Teachers with educational specialist
degree did have higher achievement gains in math and language usage than teachers with only a
bachelor’s degree (see Table 1), although schools of choice did not vary on this qualification.
Although there were differences in the work contexts of teachers across types of schools, these
contexts were not significantly related to achievement gains. The exception was that having more
limited English proficient students was associated with smaller math score gains and more
influence over school decisions was associated with larger gains in language usage. Components
of teacher professional development were not related to achievement gains in any subject.
Conclusions:
Description of conclusions and recommendations of author(s) based on findings and over study. (To support the
theme of 2009 conference, authors are asked to describe how their conclusions and recommendations might inform
one or more of the above noted decisions—curriculum, teaching and teaching quality, school organization, and
education policy.)
This study highlights the difficulty researchers have faced in identifying characteristics of
effective teachers or indicators of teacher quality. Few characteristics of teachers, their
assignments, or work contexts were related to student achievement gains. Teachers with
educational specialist degrees did have higher student achievement gains in math and language
usage, suggesting future research should explore explanations for this result. This study also
suggests that efforts to increase or regulate teacher professional development requirements may
not be an effective strategy for improving student outcomes.
2009 SREE Conference Abstract Template
5
Appendixes
Not included in page count.
Appendix A. References
References are to be in APA format. (See APA style examples at the end of the document.)
Baker, B. D., & Dickerson, J. L. (2006). Charter Schools, Teacher Labor Market Deregulation
and Teacher Quality: Evidence from the Schools and Staffing Survey. Educational
Policy, 20(5), 752-778.
Cannata, M. (2008). Teacher Qualifications and Work Environments Across School Types.
School Choice: Evidence and Recommendations Retrieved April 1, 2008, from
http://epsl.asu.edu/epru/epru_2008_Research_Writing.htm
Cannata, M. (2007). Teacher community and elementary charter schools. Education Policy
Analysis Archives, 15(11), Accessed on May 15, 2007.
Garet, Porter, Desimone, Birman, and Yoon (2001). What makes professional development
effective? Results from a national sample of teachers. American Educational Research
Journal, 38(4), 915-945.
Goe, L. (2007). The link between teacher quality and student outcomes: A research synthesis.
Washington, DC: National Comprehensive Center for Teacher Quality.
Goldring, E., & Cravens, X. (2008). Teachers' Academic Focus on Learning in Charter and
Traditional Public Schools. In M. Berends, M. Springer & H. J. Walberg (Eds.), Charter
school outcomes. New York: Lawrence Erlbaum Associates.
Hambleton, R. K. (1989). Principles and selected applications of Item Response Theory. In R. L.
Linn (Ed.), Educational measurement, 3rd edition (pp. 147-200). New York: American
Council on Education, Macmillan Publishing Company.
Harris, D. C. (2006). Lowering the bar or moving the target: A wage decomposition of
Michigan's charter and traditional public school teachers. Educational Administration
Quarterly, 42, 424-460.
Hausman, C., & Goldring, E. B. (2001). Teachers’ ratings of effective principal leadership: A
comparison of magnet and nonmagnet schools. Journal of School Leadership, 11, 399423.
Hoxby, C. M., & Murarka, S. (2008). Methods of assessing achievement of students in charter
schools. In M. Berends, M. Springer & H. J. Walberg (Eds.), Charter school outcomes
(pp. 7-37). Mahweh, NJ: Lawrence Erlbaum Associates/Taylor & Francis Group.
Ingebo, G. (1997). Probability in the measure of achievement. Chicago: MESA Press.
2009 SREE Conference Abstract Template
A–1
Kingsbury, G. G. (2003). A long-term study of the stability of item parameter estimates. Paper
presented at the annual meeting of the American Educational Research Association,
Chicago, IL.
Lord, F. M. (1980). Applications of Item Response Theory to practical testing problems.
Hillsdale, NJ: Erlbaum.
Murphy, J., Elliot, S., Goldring, E., Porter, A (2007). Leadership for Learning: A ResearchBased Model and Taxonomy of Behaviors. Journal of School Leadership and
Management. 27(2), 179-201.
Northwest Evaluation Association. (2002). RIT Scale Norm. Portland, OR: Author.
Northwest Evaluation Association. (2003). Technical Manual. Portland, OR: Author.
Podgursky, M. (2008). Teams versus Bureaucracies: Personnel Policy, Wage-Setting, and
Teacher Quality in Traditional Public, Charter, and Private Schools. In M. Berends, M.
Springer & H. J. Walberg (Eds.), Charter school outcomes. New York: Lawrence
Erlbaum Associates.
Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical Linear Models: Applications and data
analysis methods. Thousand Oaks, CA: Sage Publications.
Singer, J.D. & Willett, J.B. (2003). Applied Longitudinal Data Analysis: Modeling Change and
Event Occurrence. Oxford University Press.
National Charter School Research Project. (2007). Quantity counts: The growth of charter school
management organizations. Seattle, WA: Center on Reinventing Public Education.
2009 SREE Conference Abstract Template
A–2
Appendix B. Tables and Figures
Not included in page count.
Table 1
Achievement Growth Per Month in Math, Reading, and Language Usage
Math
Reading
Language Usage
Effect
Estimate Error Estimate
Error
Estimate
Error
Teacher characteristics and assignment
Years of experience
0.002
0.002
0.004
0.002*
0.002
0.002
Master's degree
0.025
0.028
-0.006
0.027
0.029
0.028
Education specialist degree
0.515
0.220* -0.055
0.165
3.552
0.682***
Doctorate
0.034
0.223
-0.298
0.246
0.034
0.219
Other non-BA degree as highest
degree
0.225
0.388
0.201
0.369
-0.095
0.335
Midcareer changer
-0.051
0.031
-0.007
0.029
-0.043
0.030
Attended highly selective
college
-0.015
0.058
-0.016
0.052
-0.047
0.056
Part-time teaching assignment
0.027
0.114
0.057
0.102
0.085
0.103
Permanent teaching appointment
-0.160
0.159
-0.079
0.149
-0.038
0.162
Current or former participant of
TFA or similar program
0.049
0.087
-0.034
0.081
0.006
0.081
Working conditions
Number of students per week
0.000
0.000
0.000
0.000
0.000
0.000
Percent of students that are LEP
-0.263
0.131*
0.217
0.115
0.081
0.124
Percent of students with IEP
-0.140
0.086
-0.024
0.090
0.118
0.101
Challenges: Resource shortages
-0.019
0.017
-0.018
0.015
0.021
0.016
Challenges: Teacher climate
-0.019
0.026
-0.004
0.024
-0.015
0.026
Challenges: Student workload
and needs
-0.010
0.023
-0.017
0.022
0.004
0.023
Challenges: Student
disengagement
-0.001
0.021
0.000
0.020
0.025
0.021
Influence over school decisions
0.029
0.022
0.020
0.020
0.066
0.021**
Professional development
Collective participation
0.004
0.017
0.006
0.016
-0.004
0.016
Active learning
-0.017
0.020
0.002
0.018
0.014
0.019
Reform-oriented activities
-0.008
0.015
-0.006
0.015
-0.030
0.015
Coherent
-0.023
0.017
-0.013
0.015
-0.027
0.016
1
Hours focused on content
0.000
0.001
0.000
0.000
0.001
0.000
School type
Charter
0.111
0.052*
0.184
0.046***
0.260
0.051***
Magnet
-0.073
0.073
-0.128
0.068
0.014
0.080
Private
-0.025
0.105
-0.220
0.097*
-0.229
0.093*
2009 SREE Conference Abstract Template
B–1
N
26,802
25,954
20,439
Note: All models include level 1 controls for student race-ethnicity, English Language Learner
status, special education status, free-reduced price lunch status, and grade level dummies; level 2
controls for teacher race-ethnicity and gender; and level 3 controls for school size, percent freereduced price lunch students, student race-ethnicity, and percent students Limited English
Proficient.
1
Includes the hours of professional development focused on mathematics instruction for the model
of math achievement growth and hours focused on reading/Language arts instruction for the models
of reading and language usage achievement growth.
2009 SREE Conference Abstract Template
B–2
APA Reference Style Examples
Sample Citation: Journal Article
Hypericum Depression Trial Study Group. (2002). Effect of Hypericum perforatum (St John’s
Wort) in major depressive disorder: A randomized controlled trial. JAMA, 287, 1807–
1814.
Sample Citation: Newsletter/Newspaper Article
Brown, L. S. (1993, Spring). My research with oranges. The Psychology Department Newsletter,
3, 2.
Sample Citation: Book
American Psychiatric Association. (1990). Diagnostic and statistical manual of mental disorders
(3rd ed.). Washington, DC: Author.
Booth, W. C., Colomb, G. G., & Williams, J. M. (1995). The craft of research. Chicago:
University of Chicago Press.
Sample Citation: Chapter or Section in a Book
Stephan, W. G. (1985). Intergroup relations. In G. Lindzey & E. Aronson (Eds.), The handbook
of social psychology (3rd ed., Vol. 2, pp. 599–658). New York: Random House.
Sample Citation: Web Page
Dewey, R. A. (2004). APA Style Resources by Russ Dewey. Retrieved September 8, 2004, from
http://www.psywww.com/resource/apacrib.htm
2009 SREE Conference Abstract Template