Adjusting your dreams? The effects of school
and peers on dropout behavior
Dominique Goux (CREST), Marc Gurgand
(PSE) and Eric Maurin (PSE).
The dropout problem
•
In many western countries, very significant grade retention rates and
dropout rates at the end of secondary education (OECD, 20%).
•
Universally perceived as a key issue : rising polarization, crime,
social exclusion.
•
In France, persistence of significant early dropout rates (about 12%)
although early dropout is associated with increasingly problematic
unemployment rates.
•
Large litterature describing population of dropouts, their schools and
social environment. But mechanisms leading to dropout still not
completly understood.
Economic analysis of dropout
•
Standard economic approach : anticipated rewards from
staying in school too low compared to financial and
psychological costs from doing so (Eckstein & Wolpin,1999).
•
Debated policy responses:
–
–
–
–
–
Early school intervention (Heckman, 2008).
Personal tutoring, changes in pedagogy (Dynarski et al., 2008).
Transfers/incentives (Dearden et al., 2009).
Compulsory education (Oreopoulos, 2007).
Increase perceived returns to more education (Jensen, 2010).
This paper
•
In many countries, a uniform schooling system terminates at
adolescence and gives place to a stratified system of academic and
vocational tracks.
•
Pursuing education involves making (quasi) irreversible track choices
based on own expectations about the difficulty of getting into and of
completing the different programs.
•
Many middle-school students fail to formulate realistic plans and end
up repeating grades and dropping out from school before completing
any additional year of education.
•
Working assumption : dropout decisions may be reduced by helping
families and students to define goals that are adjusted to their
academic aptitude.
The intervention
•
Paris area, 37 schools, 4,300 9th grade students, 181 classes.
•
9th grade is the end of middle school: complex and diversified supply
of tracks starting at 10th grade (high school).
•
Students apply for their preferred tracks. Admission based on
academic record.
•
Intervention where middle-school principals identify 9th graders at risk
of dropping out and help them (and their parents) to define realistic
goals.
•
Intervention randomized at the class-level, within school.
•
Administrative follow-up data on applications and assignment.
•
Detailed survey information on friendship networks within class
Results
1.
Intervention reduces early dropout rate and grade repetition by
about 1/3. It increases entry into vocational tracks.
2.
Effects persist two years after treatment. Intervention does not
just delay repetition or dropout behavior.
3.
Effects driven by more involved parents and realistic parental
aspirations.
4.
Spillover effects on non-treated students are negligible, except on
the fraction of students with both selected friends and relatively
poor academic record (compared to other non-selected).
Lessons
•
High-school dropout reduction does not necessarily involve very early
interventions, nor improvement in academic ability, nor financial help
to at-risk students.
•
Dropout can be reduced through relatively late (end of middle school)
and low cost school-based intervention.
•
Dropout can be reduced by helping low achieving middle-school
students and their families defining realistic goals,
– without over-estimating their capacity to achieve an academic track
– without under-estimating the value of « outside the box » vocational
tracks.
Roadmap
1.
Institutional context
2.
Program and design
3.
Data
4.
Effects on parents’ expectations and students’ behavior.
5.
Effects on applications and assignment
6.
Interpretation
7.
Longer term effects and spillover effects.
1. Institutional context : options after middle-school
•
At the end of middle school (9th grade), 6 possible track choices :
Within National Education :
1)
2)
3)
4)
Grade repetition (middle school),
High school, 3-year academic programme,
High school, 3-year vocational programme,
High school, 2-year vocational programme,
Outside National Education :
5) Apprenticeship (either 2-year or 3-year vocational programme),
6) Dropout.
•
Vocational system is complex :
–
–
–
In Versailles, 64 possible 3-year school programmes
46 possible 2-year school programmes
About 300 apprenticeship training centers.
1. Institutional context : applications
•
Schools decide who can be admitted to 3-year academic track
(based on performance).
•
If not admitted, students can:
– Ask to repeat 9th grade (every student is entitled with this right).
– Apply for a 2-year or 3-year vocational high school (mid June).
•
If vocational application : students are asked to list up to 4 choices
by descending order of preference.
•
Centralized Assignment System (called Affelnet) assigns as much
students as possible to one of their choices.
– 1st round : early July (about 85% studs assigned to one of their choices)
– 2nd round (more informal): mid July-Early September.
2. Program and design
•
2010-2011, largest French district (1,1 million of pupils), western
suburbs of Paris.
•
The head of the district decided to design a programme targeted at
9th grade pupils at-risk of dropping out.
•
37 middle schools ; about 9% of the 400 middle schools of the
district. Deprived schools overrepresented.
•
About 4,300 ninth graders in 179 classes (9th grade=last year of
middle school)
2. Program and design : targeted students
•
First term of AY 2010-2011: within each school and each 9th grade
class, the principal identifies pupils at risk of dropping out.
•
Early december 2010: the list is closed; about 1130 students are
selected (25% of total). On average, we have about 6 selected
pupils per class.
•
By construction, selected pupils are very different from non-selected:
– their average mark at the end of the first term is two SD smaller
– 54% have already been held back a grade (vs only 25% of other pupils).
– 33% from low income (bottom quartile) families (vs 22%)
2. Proportion of selected students by decile ranks
(pre-treatment average mark).
75
50
25
0
10
9
8
7
6
5
4
3
2
1
2. Program and design : randomization
•
Randomization : within each school, we draw at random a set of
classes where selected pupils will be invited to the programme.
•
97 test classes vs. 82 control classes.
•
Treatment and control samples are balanced
.
C
T-C
se
Obs.
Girls
44,1
-0,7
2,5
1 131
Is repeating 9th grade
7,9
-0,9
1,7
1 101
Has repeated any grade
54,5
-0,5
2,4
1 131
Low income
32,2
0,9
2,6
1 131
Average marks (/20, first term)
Maths (/20)
French (/20)
9,4
6,6
7,9
-0,02
0,04
0,10
0,14
0,21
0,26
1 094
1 070
1 060
2. Program and design : treatment vs control classes
C
T-C
se
Obs.
Class size
24.1
-0.3
0.3
179
Nb Selected students
6.3
-0.1
0.2
179
Nb Girls
12.3
-0.3
0.3
179
Nb who repeat any grade
7.8
0.2
0.4
179
Nb who repeat 9th grade
1.4
-0.1
0.1
179
Nb Low income
5.9
0.4
0.4
179
2. Program and design : the intervention
•
Two meetings between the principal and selected families. Meetings
organised at school at 18.00, January-March 2011.
•
Support provided by the district : 2 videos + methodological cards.
•
Aims of the meetings (as stated in the guidelines):
–
–
–
–
–
–
–
•
Make parents understand that an important choice has to be made.
Encourage them to get involved, help them understand procedures.
Identify families’ specific aspirations, evaluate whether they are realistic.
Provide information on alternative options.
Warn that repetition does not necessarily lead to grade improvement.
Illustrate that apprenticeship can be a solution, « outside the box ».
No information on labour market outcomes (explicitly ruled out).
The programme entails some fixed costs at the district level (DVD
conception and meetings’ guidelines), but no significant variable
costs at the school level.
2. Program and design : take up
Selected students
Non selected students
Test
Control
Test
Control
… first meeting
45,5
2,5
1,3
0,1
… second meeting
27,7
0,4
1,3
0,0
… two meetings
21,0
0,4
0,7
0,0
… one or two meetings
52,2
2,5
1,9
0,1
Obs.
600
511
1 662
1 415
3. Data
•
Survey on parents’ involvement and expectations at the end of
treatment year (response rate = 80%; balanced)
•
Administrative data on cognitive and non cognitive outcomes at the
end of the treatment year.
•
Administrative data on choices at end of treatment year.
•
Administrative data on final assignment the year after the treatment
year as well as on assignment two years after treatment.
•
Survey on students’ friendship network (response rate = 94%,
balanced).
Selection of students +
Randomization
Principal’s
intervention
Status year 1
Status year 2
Year 1
Year 2
Applications
Year 0
(2010-2011)
4. Impact on parents’ involvement and expectations
•
Parents participate much more often in information meetings at
school = direct effect of the intervention.
•
Parents participate also more often in meetings organized by
parents’ organizations and interact more often with other parents.
•
They are more often happy with school information
•
Parents form expectations about children’s ability to graduate from
high-school that become less unrealistic.
•
Dropout is not expected at all, neither in the control nor in the
treatment group. Any effect on dropout rates is likely to reflect an
effect on decisions that were not planned by families.
4. Effects on parents’ involvement and expectations
C
T-C
(se)
16.0
+24.1***
3.0
Information from school :
General information meetings
Individual meetings with career
counsellor
22.7
+1.5
2.7
Individual meetings with teachers
40.6
+2.3
3.7
Has attended meetings of parents’
organisation
9.0
+3.6*
2.1
Has talked with other parents
43.8
+9.3**
3.2
53.3
+5.7*
3.1
Baccalauréat (3-year track)
77.4
-8.2**
2.6
Vocational certificate (2-year track)
10.3
+3.4*
2.0
No diploma
0.6
-0.6*
0.3
Do not know
11.6
+5.4**
2.2
Interaction with other parents :
Satisfaction :
Happy with school information
Expected diploma :
4. Effects on parents’ involvement and expectations
•
Parents become less sure that their child will be able to graduate
from high-school.
•
In control group : 77% expect that their child will have the
baccalauréat.
•
Given the (very low) academic level of their child, this does not
seem realistic : the national graduation rate is about 67%.
•
According to administrative panel data from Ly and Riegert (2011) :
– Baccalaureat graduation if <10/20 in 9th grade = 8.2%
– Baccalaureat graduation if 10-12/20 in 9th grade = 30.0%
•
Principal’s intervention makes parents less unrealistic.
4. Effects on students’ behavior and marks
•
No significant effects on marks nor on absenteeism (or discipline)
•
The treatment does not modify pupils’ choice set by increasing the
average mark that is used by the Central Assignment System.
•
Do not succeed better at national 9th grade exam, but attend it more
often (not compulsory) = treated students are less absent on
examination day.
•
Examination day = last day of school year. Most schools provide
students with the official results of the assignment procedure just at
the end of this day.
•
Students who are absent = not interested by their own assignment.
•
In the control group, absence on examination day is a very strong
predictor of subsequent dropout behavior.
4. Effects on students’ behavior and marks
C
T-C
s.e.
Average marks, T3 (/20)
8.44
-0.00
0.11
Annual average marks (/200)
86.5
+0.7
0.9
4.2
-1.1
1.0
55.4
+1.3
2.3
Fail. but present on exam day
44.8
+6.7**
2.5
Not present on exam day
10.6
-5.4**
1.3
40.4
-0.2
2.3
1.9
+1.2
1.0
• Academic performance
• End middle school
examination
Not registered
Fail
Pass
Honors
4. Effects on applications
•
Consistent with absence of impact on academic record: the treatment
does not affect the proportion applying for 3-year academic program.
•
Consistent with impact on parents’ expectations, treatment increases
the proportion including at least one 2-year vocational track in their list
of applications (+ 4.9 points, from 15.8% to 20.7%).
•
Symmetrical decline in the proportion who either apply for 3-year
vocational program only or ask for grade repetition (or appeal).
•
Principals convinced many at risk students to adjust their applications
so as to include 2-year program in their list as a possible assignment.
•
Interpretation: principals contribute to a better perception of shorter
vocational tracks as well as to more realistic expectations about
students’ capacity to complete the longer, more selective programs.
4. Effects on applications
List of applications includes…
C
T-C
se
At least one 2-year vocational program
15.8
+4.9**
1.9
2-year vocational is first choice
11.0
+3.8**
1.7
2-year vocational is not first choice
4.8
+1.1
1.2
No 2-year vocational nor 3-year
academic programs
61.2
-5.5**
2.7
Only 3-year vocational programs
50.6
-2.5
2.7
Repetition
10.4
-3.0*
1.6
3-year academic program
16.7
+0.1
1.8
Other cases (private. other districts)
6.5
+0.5
1.1
5. Effects on assignments
•
Consistent with marks, no effect on admission into 3-year academic
programs.
•
No negative effect on entry into 3-year vocational track : treatment did
not induce students with relatively high aspirations and realistic
expectations to scale down their ambitions.
•
By contrast, treatment increases entry into 2-year vocational highschool programs (+3.3 pts) and reduces actual grade repetitions by
similar amount (-3.5 pts).
•
Decline in repetition driven by students who did not ask for repetition
whereas increase in 2-year high-school program driven by students
who put a 2-year program in their list.
•
Interpretation : by convincing students to broaden their choice list so as
to include less selective vocational tracks, principals succeed in
improving adjustment between initial choices and final assignment.
5. Effects on assignments
•
The intervention also reduces very significantly early dropout rate (40%, from 8.8 pts to 5.1 pts) and increases entry into apprenticeship
by similar amount.
•
Getting enrolled into an apprenticeship center = long and difficult
process.
•
Students have to get hired by a tutoring firm, find a seat in a relevant
training center. Interviews with potential employers typically take
place long before the end of the academic year, in March-May.
•
Increased number of apprentices not likely to reflect decisions taken
mid-July after first round of assignment procedure. Very likely to be
the outcome of a decision taken much earlier in the year.
5. Effects on assignments
C
Effet T - C
se
3-year academic
18.5
+0.0
1.8
3-year vocational
50.4
+0.8
2.9
Repetition
12.7
-3.5**
1.6
3.8
+3.3**
1.1
Apprenticeship
5.8
+3.1**
1.4
Dropout
8.8
-3.7**
1.1
Within national education
2-year vocational
Outside national education
5. Effects on assignments (cont’)
C
T-C
Se
3-year academic
18,5
+0,0
1,8
3-year vocational
50,4
+0,8
2,9
3-year vocational in the choice list
48,3
+0,3
2,9
3-year vocational not in the list
2,1
+0,5
0,8
12,7
-3,5**
1,6
Repetition in the choice list
6,5
-1,0
1,3
Repetition not in the list
6,2
-2,5**
1,0
3,8
+3,3**
1,1
2-year vocational in the choice list
3,5
+2,1**
1,0
2-year vocational not in the list
0,4
+1,2**
0,5
Apprenticeship
5,8
+3,1**
1,4
Dropout
8,8
-3,7**
1,1
• Within national education
Repetition
2-year vocational
• Outside national education
6. Interpretation
•
We observe significant effects on applications and assignments. To
what extent do they capture effects on students’ preferred tracks?
•
To what extent does our experiment contribute to the long standing
literature on the impact of « aspirations » and « expectations » on
achievement (Jacob and Wilder, 2010) ?
•
Applications and assignments depend not only on aspirations, but
also on expectations about outcomes of the assignment procedure.
•
To further interpret our results, we need additional assumptions
about how students’ applications and assignment depend on their
preferences over tracks and on expectations about assignment.
6. Interpretation : a model
•
Consider a school system where students can apply for either a 3-year
prgm (H) or a 2-year prgm (L) or repeat (R).
•
They can get access to H if their mark m is above a threshold m0
revealed at the end of the application process. By contrast, R can be
obtained at any point (even if R not in the list of applications).
•
Their preferences are driven by discounted values VH, VL and VR. No
constraints on preferences, except that if VH>VL then VH>VR
•
At the time of application, students also form expectations about m0. We
denote m0 the lowest expected value and I = (m>m0).
6. Interpretation : a model
•
A student’s application depends simply on her preferences (how she
ranks VH, VR and VL ) and her expectations about m0 (I=0 vs I=1)
•
Suppose we have data on applications. What do they identify? Do
they help identifying the effect of T on preferences.
I=0
I=1
VL > VH >VR
L
L
VH >VL>VR
L
(H,L)
VH >VR>VL
R
H
6. Interpretation : applications
•
In this set-up the negative impact of T on share of students who do
not include L in their list = a measure of the impact on the share
whose preference satisfy VH>VR>VL
•
At the end of the application process, students receive offers O which
depend on their applications and on whether actual m0 eventually
falls below or above their m.
•
A student accepts O if its value VO is above the value of outside
options : apprenticeship (VA) and dropout (VD).
•
Suppose we have data on final assignments, what do they identify?
do they help identify the type of students who have been induced to
scale down their ambitions.
6. Interpretation : assignment (cont’)
Preferences, marks and
expectations
Application
Offer
Assignment
VH >max(Vk, k=R,L,A,D)
m>max(m0,m0)
H or (H,L)
H
H
VH > VR > max(Vk, k=L,A,D)
m0>m
R
R
R
VH > VR >max(Vk, k=L,A,D)
m0>m>m0
H
R
R
L or (H,L)
L
L
VL >max(Vk, k=R,A,D)
m<max(m0, m0)
6. Interpretation : assignments
•
In this model, the fact that T decreases the share who did not ask for
repetition but were assigned to it = evidence that T induced students
with high but unrealistic expectations to scale down their ambitions.
•
By contrast, the fact that T has no effect on the share of students
assigned to H = evidence that T did not induce students with
high/realistic aspirations/expectations to scale down their ambitions.
•
The fact that T reduces the share of those who dropout after having
applied for H = evidence that T did induce some would-be dropout to
change their views over the merits of dropout vs alternative options.
•
Overall, in this set-up, empirical results are suggestive that principals
were successful in providing new goals to would-be dropouts as well
as in scaling down ambitions of students who had unrealistic ones.
7. Longer-term effects : 2 years after treatment
•
A significant fraction of students are induced to get into vocational tracks
rather than to repeat 9th grade or to dropout in year 1.
•
What are the consequence in year 2? Were selected students induced
to choose the right tracks or did the intervention only induce a delay in
grade repetition and dropout decisions?
•
In fact, two years after treatment, differences between treated and
control students tend to further increase rather than to diminish :
– Dropout gap = -5.1 pts in year 2 (versus -3.8 pts in year 1)
– Grade advancement gap = -4.4 pts in year 2 (versus -4.1 pts in year 1)
•
No evidence that principals contributed to distort students’ perceptions
in favour of tracks that did not correspond to their abilities/aspirations.
7. Longer-term effects : 2 years after treatment
C
T–C
se
52.7
+4.4*
2.6
3-year general (2nd year)
10.8
+1.0
1.6
3-year vocational (2nd year)
38.6
-0.0
2.7
2-year vocational (2nd year)
3.3
+3.4**
1.0
First year completed
First year still not completed
20.0
-1.9
2.2
3-year general (1st year)
9.0
-1.0
1.5
3-year vocational (1st year)
9.2
-0.9
1.6
2-year vocational (1st year)
1.7
+0.0
0.7
7.3
+2.3*
1.4
20.0
-5.1**
1.9
Dropout in year 1
7.5
-3.9**
1.0
Repetition in year 1
2.3
-1.5**
0.6
10.2
+0.2
1.6
Apprenticeship
Dropout
Others
7. Longer-term effect : end of the last 9th grade
•
We also looked at outcomes at the end of the last 9th grade : first 9th
grade if did not repeat, but second 9th grade if repeated.
•
No difference in marks: no evidence that those induced not to repeat
would have benefited from an additional year in middle school. Many
repetitions not useful and principals are able to detect/avoid them.
Moyenne C Effet T - C
se
Obs.
• End of last 9th grade :
Average marks (/240)
87.7
+0.6
0.9
1102
Prop. whose average
marks>110
14.9
+0.9
2.0
1102
Proportion admitted in general
track
22.7
-0.9
2.0
1109
7. Longer term effects on dropout : a summary
•
Overall, intervention contributed to reducing dropout through two main
channels.
•
First, it helped a fraction of students initialy willing to dropout to define
new prospects.
•
They turn out to be able to persist in education and to complete one
additional year of education.
•
Second it induced a fraction of students initially willing to repeat 9th
grade to apply for and go into 2-year programs.
•
Most of them turn out to be able to complete one additional year of
schooling whereas they would have ended up dropping out from school
had they not been treated.
8. Spillover effects on non selected students
1.
Very small average effect of intervention on applications or status
one year after treatment for non selected students.
2.
The majority of non selected students (60%) have no selected
friends. Also the majority is admitted into academic track: no room
for peer group influence.
3.
When we focus on the 20% with selected friends and relatively
modest academic record: significant increase in entry into
vocational schools (at the detriment of academic schools).
4.
The rehabilitation of vocational tracks by principals induced a
significant fraction of non-selected students to get into these
specific tracks rather than to pursue into the general track.
5.
However, this spillover effect tend to fade out in year 2 : those
induced to get into vocational track in year 1 would have got into
this track one year later, had they not been treated.
8. Spillover effects on non selected students:
applications
All
C
Appeal or repetition
Some selected
friends
T-C
C
T-C
Some selected
friends and pretreatment marks<12
C
T-C
3,1
+0,9 (0,7)
4,3
+0,2 (1,1)
9,2
-1.1 (2,3)
3-year general
73,2
-1,0 (1,4)
67,7
-3.6 (2,3)
37.8
-7.7 (3.6)**
3-year vocational
22,2
-0,0 (1,4)
28,4
+2,4 (2,3)
51,7
+8.7 (3.8)**
2-year vocational
6,2 -2,0 (0,9)**
6,2
+0,3 (1,3)
13,0
-2.2 (2.6)
Others
5,1
3,4
+0,8 (0,8)
5,5
+0.7 (1.4)
Obs.
+0,1 (0,6)
2 972
1 208
528
8. Spillover effects on non selected students: status one
year after the treatment
Some selected
friends
All
Some selected
friends and pretreatment
marks<=12
C
T-C
C
T-C
C
T-C
Repetition
2.7
+0,5 (0.6)
3.9
-0.5 (1.0)
8.8
-1.2 (2.3)
3-year academic track
73.6
-2.0 (1.4)
67.9
-4.1 (2.3)*
39.1
-7.4 (3.4)**
Vocational tracks
19.5
+1.6 (1.5)
24.1
+3.8 (2.2)
45.4
+9.2 (3.9)**
3-year vocational
16.1
+1.3 (1.4)
20.7
+3.1 (2.2)
39.5
+6.1 (4.3)
2-year vocational
2.0
-1.0 (0.5)**
1.9
-0.2 (0.7)
3.8
-0.3 (1.5)
Apprenticeship
1.4
+0.5 (0.4)
1.5
+0.9 (0.8)
2.1
+2.1 (1.5)
4.2
+0.8 (0.7)
4.1
+0.7 (1.1)
6.7
-0.1 (1.9)
Dropouts
Obs.
2 972
1 208
529
9. Concluding remarks
•
School principals are able to help students at risk of dropping out
to form more realistic expectations and aspirations.
•
It induces about 30% of would-be dropouts to stay longer in
education and successfully complete additional years of
education/training.
•
In a partial population design, spillovers may be difficult to detect
without information on actual interactions between eligible and
non-eligible.
Institutional context : labour market
•
Dropout (especially early dropout) associated with very poor labour
market outcomes in France.
Unemployment
Monthly wages
3-year general
10.6
1354.0
1.66
3-year vocational
14.0
1090.5
1.34
2-year vocational
24.1
967.7
1.19
Apprenticeship
19.5
1020.7
1.25
Dropout
Late dropout
Early dropout
42.8
36.3
49.5
865.0
915.8
815.8
1.06
1.12
1.00
Medium-term effect : interpretation
•
Treatment affects status in year 1 through two different processes,
one affecting students who intended to leave education, one
affecting students who intended to pursue education.
•
For the first group, principals convinced students to scale down their
ambitions. As a results, they obtain one of their listed choice more
often and end up repeating less.
•
For the second group, they convinced students that apprenticeship
is a better way to get a foothold in the workplace (as well as to get
additional education) than direct entry into the labor market.
•
This interpretation has testable implications: no effect of intervention
on the size and composition of the group of pupils who stay in the
education system (nor on those of the leavers).
Medium-term effect : interpretation
Moyenne C
Stay in education system
Effet T-C
s.e.
Obs
85,4
+0,7
1,6
1 130
55,4
+0,5
2,8
971
6,8
-0,7
1,7
971
Low income
32,9
+1,8
2,9
971
Annual verage marks (/240)
87,5
+0,8
0,8
971
Average marks first term (/20)
9,29
-0,06
0,08
971
Conditional on education system
Boys
Has just repeated 9th
Interpretation : early dropout
•
In this model, dropouts include those who prefer dropout to any
other options (VD=maxVk).
•
But it also include students who prefer H to D but have been
induced to dropout by either poor marks or over pessimistic
expectations (i.e., by either m<m0 or m<m0 ).
•
ID result 4: if the negative impact on T on early dropout is driven by
change in expectations only, we should observe a decline in the
share of students who dropout after having applied to L or R only.
•
This is not what we observe : dropout decline is mostly driven by
those who dropout after having expressed optimistic expectations
(i.e., H in their list).