Effects of Poverty, Funding
Structure and Scale on Public
School System Performance
John Mackenzie
FREC/CANR, University of Delaware
May, 2010
BACKGROUND:
Coleman Report (1966): finds link between money and
school performance to be tentative at best.
(Basic problem: lack of data!)
Eric Hanushek (1981,1986,1997): “Throwing Money at
Public Schools” meta-analyses of studies relating
funding to school performance.
(Methodological error in counting studies as datapoints:
7 heads in 10 coin tosses does not prove a coin is biased, but
combining 100 trials of 10 tosses and getting 7 or more heads
in 60% of the trials does.)
Jay Greene (Manhattan Institute): Real per-pupil spending
“almost doubled” from 1972 to 2002, while NAEP scores did
not improve much at all.
(So what? Real per-capita disposable incomes “almost
doubled” too. Compare rising costs of college!)
Public schools today are…
more inclusive of minorities, immigrants, etc.;
offer a broad array of non-traditional services;
serve an expanded population of poor children;
deliver more remedial & special education.
Education is typically a “luxury” good: as incomes rise,
households invest larger proportions of their incomes in
education. Education confers status; may be a positional good.
US income inequality continues to increase, and the relative
economic return to a HS diploma is falling.
If education lifts families from poverty for multiple generations,
is residual poverty becoming more intractable? Does the rising
real cost of public schools simply reflect the rising marginal cost
of eliminating poverty?
School Spending/Pupil vs. Median Household Income, by State
Is Public Education a “Luxury” Good?
$14,000
y = 0.1242x + 2793.5
K-12 Public School Spending/Pupil, 2004
R2 = 0.2252
$12,000
US: $8,287 per pupil
$44,231 median income
Elasticity = +0.66
$10,000
$8,000
$6,000
$4,000
$30,000
$35,000
$40,000
$45,000
Median Household Income, 2003
$50,000
$55,000
$60,000
A favorite neo-con hobbyhorse: “Throwing Money at Schools”
SAT05 vs. Total 2004 Per-Pupil Spending, by State
1250
1200
1150
1100
1050
1000
950
$4,000 $5,000 $6,000 $7,000 $8,000 $9,000 $10,000 $11,000 $12,000 $13,000 $14,000
NAÏVE UNIVARIATE MODELS [t-statistics in brackets]
SAT02 = 1187.16 – 0.0156*TXPP02
{25.70]
[-2.59]
N=50; R-square=0.123
SAT03 = 1185.38 – 0.0143*TXPP03
[26.06]
[-2.51]
N=50; R-square=0.116
SAT04 = 1181.65 – 0.0131*TXPP04
{26.85]
[-2.49]
N=50; R-square=0.114
SAT05 = 1196.88 – 0.0117*TXPP05
[28.45]
[-2.87]
N=50; R-square=0.146
Those kids get
dumber every
year, Jill!
They sure
do, Bob!
Is my hair
OK?
More waste and bloat in public education?
School spending rises and SAT scores fall!
State mean SAT scores depend on test participation rates
1250
2004 SAT verbal+math
1200
1150
SAT04 = 993.23 - 59.574Ln(%Participation)
2
R = 0.8435
1100
1050
1000
950
0%
20%
40%
60%
Percent of seniors taking the SAT in 2004
80%
100%
A textbook example of an omitted variable problem:
REGRESSION MODELS CONTROLLING FOR PARTICIPATION
SAT02 = 1062.5 + 0.0135*TXPP02 – 244.58*Partic02
[43.29] [3.46]
[-12.82]
N=50; R-square=0.806
SAT03 = 1074.5 + 0.0113*TXPP03 – 234.60*Partic03
[43.29] [3.46]
[-12.82]
N=50; R-square=0.809
SAT04 = 1079.4 + 0.0104*TXPP04 - 230.50*Partic04
[48.65] [3.37]
[-13.00]
N=50; R-square=0.807
SAT05 = 1095.9 + 0.0070*TXPP05 - 221.17*Partic05
[46.60] [2.63]
[-11.47]
N=50; R-square=0.775
ECONOMETRIC DIGRESSION
A formal treatment of the participation bias problem:
When only high-performing students take the SAT the mean score is
biased upward. Expanding participation to include more typical students
educes the bias.
Low SAT participation boosts the mean scores of
low-performing states above the mean scores of
high-performing states.
2005 ACT Participation vs. 2005 SAT Participation, by State
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
2005 SAT Participation vs. Total 2004 Per-Pupil Spending, by State
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
$4,000
$6,000
$8,000
$10,000
$12,000
$14,000
$16,000
FORMAL HECKMAN MODEL:
PROBIT: NORMSINV(Partic%05) = ExpPP04, ACTPart%05
Regression Statistics
Multiple R
0.9153
R Square
0.8378
Adjusted R Square
0.8309
Standard Error
0.3970
Observations
50
ANOVA
df
Regression
Residual
Total
Intercept
ExpPP04
ACTPart%
SS
38.245
7.406
45.651
MS
19.122
0.158
F
121.357
Coefficients Standard Error
-0.093569
0.374798
0.000107
0.000038
-2.531849
0.211647
t Stat
-0.249651
2.790748
-11.962623
P-value
0.803945
0.007576
0.000000
2
47
49
PROBIT Residuals
State
Alabama
Alaska
Arizona
Arkansas
California
Colorado
Connecticut
Delaware
Florida
Georgia
Hawaii
Idaho
Illinois
Indiana
Iowa
Kansas
Kentucky
Louisiana
Maine
Maryland
Predicted
-1.34361
Residuals
0.06205
Lambda
0.17768
PredPart%
9.0%
0.32769
0.06969
-1.29832
0.37901
-1.80895
0.80478
0.89688
-0.40756
-0.00240
0.41217
-0.91858
-1.67618
0.25853
-0.95007
-1.21531
-1.28251
-1.47621
0.67091
0.58593
-0.27753
-0.50960
-0.25645
-0.37901
1.16560
0.27554
-0.25354
0.79288
0.67689
-0.13285
0.11216
0.39463
0.15393
-0.69479
-0.12544
0.10753
0.07114
0.00358
-0.03254
1.01753
0.84277
0.19021
1.05379
0.08052
1.37111
1.44319
0.55781
0.79636
1.07747
0.31873
0.10272
0.96943
0.30646
0.21470
0.19472
0.14428
1.26839
1.20453
62.8%
52.8%
9.7%
64.8%
3.5%
79.0%
81.5%
34.2%
49.9%
66.0%
17.9%
4.7%
60.2%
17.1%
11.2%
10.0%
7.0%
74.9%
72.1%
SAT05PCT
10%
52%
33%
6%
50%
26%
86%
74%
65%
75%
61%
21%
10%
66%
5%
9%
12%
8%
75%
71%
Lambda = NORMDIST(PredY,0,1,FALSE)/(1-NORMDIST(PredY,0,1,TRUE))
= f(X)/[1-F(X)] where f and F are standard normal density and distribution
2005 SAT %Participation, by State -- Actual vs. Predicted from Probit
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Predicted & Actual Participation vs. LAMBDA
1.80
1.60
1.40
1.20
1.00
0.80
0.60
0.40
0.20
0.00
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Incorporate LAMBDA as participation bias correction instrument:
Heckman 2nd Stage: SAT05 = ExpPP04, Lambda
Regression Statistics
Multiple R
0.8222
R Square
0.6760
Adjusted R Square
0.6622
Standard Error
Observations
38.9339
50
ANOVA
df
SS
MS
F
2
148622.07
74311.03
49.02
Residual
47
71244.91
1515.85
Total
49
219866.98
Coefficients
Std Error
t Stat
P-value
1054.6446
30.8222
34.2170
0.0000
ExpPP04
0.0148
0.0045
3.3095
0.0018
LAMBDA
-145.1194
15.9954
-9.0726
0.0000
Regression
Intercept
Or construct the bias correction instrument
directly from the observed participation rates:
Fast Heckman: SAT05 = ExpPP04, L*
L* = (NORMDIST(NORMSINV(%Partic05),0,1,FALSE)/(1-%Partic05)
Regression Statistics
Multiple R
0.86749
R Square
0.75254
Adjusted R Square
0.74201
Standard Error
34.02397
Observations
50
ANOVA
Regression
Residual
Total
Intercept
ExpPP04
Heckman
df
2
47
49
SS
165458.35
54408.63
219866.98
MS
82729.17
1157.63
F
71.46
Coefficients
1064.9402
0.0128
-132.3417
Std Error
26.1859
0.0037
11.9657
t Stat
40.6684
3.4751
-11.0601
P-value
0.0000
0.0011
0.0000
Or use the equivalent logit-based instrument (McFadden):
Fast McFadden: SAT05 = ExpPP04, MU
where MU = -%Part05*ln(%Part05)/ln(1-%part05) - ln(1-%Partic05)
Regression Statistics
Multiple R
0.85742
R Square
0.73517
Adjusted R Square
0.72390
Standard Error
35.19792
Observations
50
ANOVA
Regression
Residual
Total
Intercept
ExpPP04
MU
df
2
47
49
SS
161639.0
58228.0
219867.0
MS
80819.5
1238.9
F
65.2
Coefficients
1060.261
0.01335
-74.0986
Std Error
27.343
0.00386
7.0262
t Stat
38.777
3.45532
-10.5460
P-value
0.0000
0.0012
0.0000
Fast McFadden vs. Fast Heckman
4.00
3.50
y = 1.7858x - 0.0017
R2 = 0.9985
3.00
2.50
2.00
1.50
1.00
0.50
0.00
0.00
0.20
0.40
0.60
0.80
1.00
1.20
1.40
1.60
1.80
2.00
“50 experiments in public school system finance
and structure”
Data Sources:
US Census Bureau, Public Elementary–Secondary
Education Finance Data (annual State-level tables)
http://www.census.gov/govs/school
US Dept. of Education, 2003, 3005, 2007 and 2009
National Assessments of Education Progress (NAEP)
http://nces.ed.gov/nationsreportcard/statecomparisons/
NAEP09 vs. Total 2007 Per-Pupil Spending, by State
1080
1060
1040
1020
1000
y = 0.0035x + 967.43
2
R = 0.1343
980
960
940
$6,000
$8,000
$10,000
$12,000
$14,000
$16,000
$18,000
$20,000
NAEP09 vs. 2007 FEDERAL Funding per Pupil, by State
1080
1060
1040
1020
y = -0.0312x + 1038.3
R2 = 0.1488
1000
AK
980
LA
960
940
$500
$700
$900
$1,100
$1,300
$1,500
$1,700
$1,900
$2,100
$2,300
NAEP09 vs. 2007 STATE Funding Per Pupil, by State
1080
1060
VT
1040
1020
1000
y = -5E-05x + 1007.4
2
R = 0.00002
980
HI
960
940
$2,000
$4,000
$6,000
$8,000
$10,000
$12,000
$14,000
$16,000
NAEP09 vs. 2007 LOCAL Funding per Pupil, by State
1080
1060
MA
NJ
CT
1040
PA
1020
NY
1000
y = 0.0062x + 978.52
2
980
R = 0.2697
960
940
$0
$2,000
$4,000
$6,000
$8,000
$10,000
$12,000
NAEP09 vs. 2007 FEDERAL, STATE and LOCAL Per-Pupil Funding, by State
Regression Statistics
Multiple R
0.62032
R Square
0.38480
Adjusted R Square
0.34468
Standard Error
20.55471
Observations
50
ANOVA
df
Regression
Residual
Total
Intercept
FED07PP
STATE07PP
LOCAL07PP
SS
12156.15
19434.82
31590.97
MS
4052.05
422.50
Coefficients
Standard Error
992.89010
15.28438
-0.02583
0.00964
0.00211
0.00137
0.00599
0.00144
t Stat
64.96109
-2.67844
1.54618
4.16168
3
46
49
F
9.59
P-value
0.00000
0.01022
0.12891
0.00014
NAEP09 vs. Percent of Students Eligible for FRPL, by State
1080
1060
1040
1020
y = -249.32x + 1112
1000
R2 = 0.8076
980
960
940
920
20%
30%
40%
50%
60%
70%
80%
2009 NAEP
4th grade
MATH
2009 NAEP
8th grade
MATH
2009 NAEP
4th grade
READING
2009 NAEP
8th grade
READING
NAEP vs. Log2(Avg. District Size), by State
1080
y = -7.123x + 1090.6
1060
2
R = 0.2322
1040
1020
1000
980
960
940
6
8
10
12
14
16
18
State funding
replacing
property tax
Targeted
Federal
dollars
(Title I, etc.)
Less reliance
on property
taxes in larger
districts
Less reliance on
local funding in
higher-poverty
states
Larger average
district size in
higher-poverty
states
NAEP08 vs. Per-Pupil Funding by Source and Log2 of Avg. District Size
Regression Statistics
Multiple R
0.7959
R Square
0.6335
Adjusted R Square
0.5918
Standard Error
16.2225
Observations
50
ANOVA
Regression
Residual
Total
Intercept
FED07PP
STATE07PP
PROPTX07PP
OTHRLOC07PP
L2AVDISTSZ
df
5
44
49
SS
20011.50
11579.47
31590.97
MS
4002.30
263.17
F
15.21
Coefficients Standard Error
1089.5879
21.7298
-0.0318
0.0077
0.0028
0.0011
0.0046
0.0015
0.0059
0.0013
-7.7268
1.4171
t Stat
50.1425
-4.1269
2.5162
3.1430
4.6607
-5.4526
P-value
0.0000
0.0002
0.0156
0.0030
0.0000
0.0000
NAEP08 vs. Federal, State, Property Tax and Other Local Funding, Percent Poverty and log2 of Dis
of Avg. District Size
Regression Statistics
Multiple R
R Square
Adjusted R Square
Standard Error
Observations
0.9315
0.8676
0.8491
9.8625
50
ANOVA
df
Regression
Residual
Total
Intercept
FED07PP
STATE07PP
PROPTX07PP
OTHRLOC07PP
PCTPOV
L2AVDISTSZ
SS
27408.45
4182.52
31590.97
MS
4568.07
97.27
Coefficients
Standard Error
1127.24317
13.89842
-0.00488
0.00561
0.00061
0.00071
0.00129
0.00096
0.00241
0.00087
-197.56627
22.65538
-3.74497
0.97503
t Stat
81.10587
-0.87019
0.85619
1.34205
2.77650
-8.72050
-3.84086
6
43
49
F
46.96
P-value
0.00000
0.38903
0.39664
0.18662
0.00811
0.00000
0.00040
Similar results are obtained with 2007, 2005 and 2003
NAEP scores analyzed against contemporaneous or
lagged funding, average district size, poverty, etc.
(These are large, slowly-evolving systems)
Poverty is the strongest predictor of NAEP performance,
but is collinear with federal funding (+), local funding (-)
and average district size (+).
Structural Variables: Federal funding is mainly targeted to
low-performing populations—Title I and special needs.
State funding does not drive NAEP performance much at all.
Local funding drives NAEP performance most efficiently.
Average district size is negatively correlated with NAEP
performance, suggesting scale diseconomies.
Implications of funding and average district size:
HI, FL*, MD*, NV, UT, NC, LA: large average district size,
generally more state funding, lower NAEP scores
VT*, MT, ND, ME, SD, NE, OK, NH: many small districts,
generally more local funding, higher NAEP scores
Likely sources of public dissatisfaction with schools:
Displacement of local funding with state funding, and
consequent loss of local control.
Legacy of desegregation, Serrano, consolidation,
bureaucratization, political intervention and
accumulation of regulation, union protection of teacher
seniority, etc.
These results suggest the efficiency of public
education can be improved by restoring local
autonomy of school systems.
Why would local dollars drive school system
performance more efficiently than state dollars?
Local funding implies stronger local governance and
accountability.