The economic returns to a degree:
how great and how varied are they?
(Is there are an earnings premium for a first or a 2.1?)
Gianna Boero, Robin Naylor, and Jeremy Smith
University of Warwick
UoH 27th April 2016
1
Plan of Talk
1.
Context 1:
Evidence and Policy
Context 2:
Theory and Interpretation
2.
Institutional Arrangements
3.
Data and Methodology
4.
Results
5.
Conclusions and Further Work
UoH 27th April 2016
2
Plan of Talk
1.
Context 1:
Evidence and Policy
Importance of HE
Human Capital, R&D, Economic Growth
HE Participation and Labour Supply
Socio-economic Mobility/Persistence
Political Economy: fees and funding
Returns to Education
Years of Schooling
Qualification Levels
Grades Performance
HE Policy relevance of estimated returns in UK
Dearing Report and evidence from Blundell et al. (2000: PTO)
Browne Report
UoH 27th April 2016
3
Average Graduate Premium in UK
Blundell et al. (2000):
NCDS1958 birth cohort
1991 hourly wage data
Estimates
ln wi   HEi   X i   i
where HE is Highest Educational Qualification
Rich set of observable characteristics
Assumes that:
E ( i | HEi , X i )  E ( i | X i )
Individuals with different HE do not
differ on average in unobservables.
Results
Graduate Earnings Premium
(Relative to control group with 2+ A-levels)
17%
37%
Men
Women
UoH 27th April 2016
4
Variation around Average
Subject
Higher for Science, Social Science
(Harkness and Machin, inter alia)
Institution
Higher for ‘Elite’ HEIs
(Chevalier et al., inter al.)
Hence Differential Fees
(Greenaway and Haynes)
Prior Schooling
(Naylor and Smith)
Cohort
Walker and Zhu, 2008
Expansion no effect on average
Increased premium in highest quartile
(Ability composition effect?)
… Degree Class Premia
(Not available in NCDS)
UoH 27th April 2016
5
Interpretation of Graduate Earnings Premium
Human Capital Theory
Signalling/Screening/Sorting Theories
Statistical Discrimination
Short-run only? Employer Learning/Statistical Discrimination
UoH 27th April 2016
6
Jo Johnson, Minister of State for Universities and Science, speaking about
BIS’s Green Paper ‘Fulfilling our Potential: Teaching Excellence, Social
Mobility and Student Choice’, stated that,
“We want to encourage a (grade point average) system which provides
greater information to employers about where attainment really lies. It
needs to sit alongside, rather than replace, the honours degree
classification… But there is a very big band, the 2.1 band. It disguises very
considerable differences in attainment. You can be at the top of the band
and then be 50 percentage points below and still be getting a 2.1. And
students who worked hard should be able to signal to employers that’s
what they’ve achieved.”
(Cited on BBC News website 06/11/2015.)
UoH 27th April 2016
8
Degree Class Premia: Evidence from available data
BCS70
LFS
USR/HESA
GCS
Background
Classification of Honours Degrees:
First
Upper Second
(>=2.1 => ‘Good’)
Lower Second
(=<2.2 => ‘Lower’)
Third
Pass (Non-honours)
Classification Rules:
Based on:
Overall average
Papers in class
Final exams/coursework
Viva
Anecdotally:
‘Achieves’ vs ‘Is’!
= HKT vs Signalling!
UoH 27th April 2016
9
Estimated log wage premia (BCS70)
(1)
(2)
(3)
(4)
(5)
Wages observed in year:
2000
2000
2000
2000
2000
Wages observed at age:
30
30
30
30
30
Good degree class premium
relative to lower degree class
0.078
(0.007)
0.077
(0.008)
0.073
(0.012)
0.071
(0.014)
0.068
(0.019)
Lower degree class premium
relative to 2+ A-levels
0.119
(0.000)
0.105
(0.001)
0.107
(0.000)
0.103
(0.001)
0.109
(0.000)
Family background
No
Yes
Yes
Yes
Yes
Ability at age 10
No
No
Yes
Yes
Yes
Ability at age 5
No
No
No
Yes
Yes
Non-Cognitive ability at ages 5, 10
No
No
No
No
Yes
Other controls
Yes
Yes
Yes
Yes
Yes
3046
3046
3046
3046
3046
0.0814
0.0988
0.1029
0.1119
0.1188
No. of Obs
R2
Good
degree
premium
over
Lower
Lower
degree
premium
over Alevels
Notes: p-values in parentheses. Ability controls include: BAS (verbal), BAS (numerical). Background controls include: parental income, parental social class,
mother’s interest in education, father’s interest in education, mother’s education, father’s education. Other controls include: region (aged 10), gender,
marital status and number of children, ethnicity.
UoH 27th April 2016
10
Estimated log wage premia (LFS): selected birth cohorts in 1969-1971
Wages observed at:
2005-2012
Wages observed at age:
36-41
Good degree class premium
(relative to lower degree class)
0.087
(0.001)
0.188
Lower degree class
(0.000)
(relative to 2+ A-levels)
Yes
Other controls
No. of Obs
2930
R2
0.152
UoH 27th April 2016
11
Estimated log-earnings premia (USR91, graduate cohort), birth cohort
1969-1971
USR-FDS:
First Destination
Median Occupational earnings
Earnings observed at:
1992
1992
Earnings observed at age:
21-23
21-23
Good degree class premium
relative to lower degree class
0.046
(0.000)
0.043
(0.000)
Ability and background controls
No
Yes
Other controls
Yes
Yes
No. of Obs.
22,459
22,459
R2
0.334
0.336
Note: p-values in parentheses. Ability controls include: pre-University qualifications. Background controls include: social class
of parents, school-type. Other controls include: gender, marital status, University attended and type of degree course.
UoH 27th April 2016
12
Estimated log-wage premia (GCS1990, graduate): birth cohort 1968-1970
(1)
(2)
(3)
(4)
Wages observed at:
1991
1991
1996
1996
Wages observed at age
21-23
21-23
26-28
26-28
Good degree class premium
relative to lower degree class
0.051
(0.014)
0.049
(0.014)
0.084
(0.014)
0.079
(0.014)
Ability and background controls
No
Yes
No
Yes
Other controls
Yes
Yes
Yes
Yes
No. of Obs
2839
2839
3652
3652
R2
0.127
0.131
0.115
0.119
Note: p-values in parentheses. Ability controls include pre-university qualifications, background controls include parental
education, and other controls include age, gender, ethnicity, and marital status.
UoH 27th April 2016
13
Figure 5a: Coefficients on degree class variables over time ( constant earnings) - Males
0.08
0.06
0.04
0.02
Coeff
0.00
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
-0.02
-0.04
-0.06
-0.08
-0.10
-0.12
Year
First
2:2
Third
UoH 27th April 2016
Other
15
Plan of Talk
1.
Context 1:
Evidence and Policy
Context 2:
Theory and Interpretation
How might we interpret evidence of a premium by class of degree
awarded?
Why might any premium by degree class change across cohorts?
UoH 27th April 2016
16
Hypothesis 1
Pay
But if Econometrician
observes only Degree
Class, then there
appears to be a
Premium by Class:
might wrongly interpret
this as a discontinuity.
280
PAYPROVA
240
200
160
120
80
0
20
40
60
80
100
Average Mark
MARK1
UoH 27th April 2016
19
A Regression Discontinuity framework offers the prospect of being able to
distinguish between the two hypotheses – but requires us to observe both degree
classification and underlying marks.
Hypothesis 2
Pay
Employer regards
Degree Class as a Signal
of some dimension of
ability.
A discontinuity would be
indicative of signalling or
statistical discrimination
in the sense of the EL-SD
approach
I
a
Average Mark
UoH 27th April 2016
23
Regression discontinuity and degree class effects
Also see: di Pietro (2012);
Feng and Graetz (2015)
We use individual student data on an anonymous university located somewhere
near the centre of England…
Anonymised DLHE returns for graduate cohorts of 2011/12, 2012/13, 2013/14.
Matched by personal id to extensive individual student records.
Data include:
Age, gender, nationality, fees status, course, department, previous schooling, family
background, degree class, marks per module per year.
Labour market outcome, 5-digit SOC, SIC, salary, degree class, location…
UoH 27th April 2016
30
3.
Data and Methodology
To date, we have data only for individuals who have responded to the DLHE in each
cohort – we are waiting student records on all students in order to establish the
extent to which DLHE respondents might differ in observable characteristics from
non-respondents.
The DLHE response rate is approximately 63%.
Results to be presented today exploit data on graduates who are in full-time
employment and have provided Research-Accessible personal salary information.
The usable response rate to the salary question is approximately 41%.
UoH 27th April 2016
34
Population of All Leavers
n= 25950
DLHE Respondents
n= 16360 (63%)
Online/Post
n= 6751 (41%)
PG
n= 3515(52%)
Telephone
n= 9609 (59%)
UG
n= 3265 (48%)
FT-Employed
n= 1691 (52%)
Further Study
n= 931 (29%)
OLFU
n= 6751 (19%)
Salary Data
n= 1404 (83%)
UoH 27th April 2016
36
UG Students. Degree Class breakdown by:
All DLHE Respondents, FT-Emp, Salary Info
Degree
DLHE
In FT
Salary
Class
Resp
Emp
Info
-------------------------------------------------1st
35%
34%
35%
2:1
53%
54%
53%
2:2
11%
11%
11%
3rd
2%
2%
2%
-------------------------------------------------n
3265
1691
UoH 27th April 2016
1404
38
UG
n= 3265
UG HEU+3/4-yr+08-11start
n= 2791
FT-Employed
n= 1691 (52%)
FT-Employed
n= 1517 (54%)
Salary Data
n= 1404 (83%)
Salary Data
n= 1292 (85%)
Analysis will be based on a sample which excludes OS students,
any students starting later than 11/12 and any students on UG
degrees other than 3 or 4 year duration.
UoH 27th April 2016
40
Probability of receiving Treatment (= Good degree)
Cut-off at 60
All UGs in responding to DLHE online/post:
(except: OS, course duration ~=3|4)
Scatter Plot: Bins=30
0
.2
.4
good
.6
.8
1
Based on n=2705
50
60
70
overall_ave
UoH 27th April 2016
80
90
41
Probability of receiving Treatment (= First)
Cut-off at 70
All UGs in responding to DLHE online/post:
(except: OS, course duration ~=3|4)
Scatter Plot: Bins=30
0
.2
.4
first
.6
.8
1
n=2705
50
60
70
overall_ave
UoH 27th April 2016
80
90
43
Compliers and Non-compliers
All: 2.1 versus 2.2
n=1707: Compliers=92%
2.2
2.1
<60
281
136
All: 1st versus 2.1
n=2422: Compliers=89%
>=60
2
1288
2.1
1
Science: 2.1 versus 2.2
n=751 : Compliers=94%
<60
>=60
2.2
201
1
2.1
46
503
UoH 27th April 2016
<70
1422
272
>=70
2
726
Science: 1st versus 2.1
n=1091 : Compliers=90%
<70
>=70
2.1
548
1
1
107
435
45
10
8
Percent
6
4
2
0
8
9
10
lannualpay
11
12
Density of log(annual pay))
UoH 27th April 2016
47
9.5
10
10.5
RD Plot: cutoff=60
40
60
Sample average within bin
80
100
4th order global polynomial
Data-driven RD plots
See Calonico, Catteneo and Titunik (2014)
Bin scatter plot of log(annual pay)
versus Overall Average Mark
All Cut-off 60
n=1193
UoH 27th April 2016
49
9.7
9.8
9.9
10
10.1
10.2
RD Plot: cutoff=60
50
55
60
Sample average within bin
65
70
4th order global polynomial
Data-driven RD plots
See Calonico, Catteneo and Titunik (2014)
Bin scatter plot of log(annual pay)
versus Overall Average Mark
Science Cut-off 60
n=646
UoH 27th April 2016
51
9.8
10
10.2
10.4
RD Plot: cutoff=70
50
60
Sample average within bin
70
80
90
4th order global polynomial
Data-driven RD plots
See Calonico, Catteneo and Titunik (2014)
Bin scatter plot of log(annual pay) versus
Overall Average Mark
IMSE-optimal evenly spaced method using
polynomial regression (espr)
All Cut-off 70
n=1193
UoH 27th April 2016
52
9.8
10
10.2
10.4
RD Plot: cutoff=70
50
60
Sample average within bin
70
80
90
4th order global polynomial
Data-driven RD plots
See Calonico, Catteneo and Titunik (2014)
Bin scatter plot of log(annual pay) versus
Overall Average Mark
IMSE-optimal quantile-spaced method using
spacings estimators (qs)
Science Cut-off 70
n=646
UoH 27th April 2016
53
-.5
0
.5
1
RD Plot: cutoff=60
40
60
Sample average within bin
80
100
2th order global polynomial
Continuity (Gender)
(RDPLOT)
All P(2)
UoH 27th April 2016
54
-.5
0
.5
1
RD Plot: cutoff=60
40
60
80
Sample average within bin
100
4th order global polynomial
Continuity (FT-Employment)
Cut-off 60
(RDPLOT)
ALL P(4)
UoH 27th April 2016
56
0
.2
.4
.6
.8
RD Plot: cutoff=70
40
60
Sample average within bin
80
100
4th order global polynomial
Continuity (FT-Employment)
Cut-off 70
(RDPLOT)
ALL P(4)
UoH 27th April 2016
58
0
.5
1
RD Plot: cutoff=70
40
60
Sample average within bin
80
100
4th order global polynomial
Continuity (FS)
Cut-off 70
(RDPLOT)
ALL P(4)
UoH 27th April 2016
59
3.
Data and Methodology
Issue of manipulation/precision of control over assignment variable
Student manipulation:
Those on track for ‘borderline’ after Year 2 work hard to achieve 2.1,
those below borderline reduce effort. So we’d see a trough in the density
distribution.
Note: manipulation of overall average much harder than of a single module
Marker manipulation:
Eg practice of avoiding ‘9s’
Exam Board manipulation:
If Board uses ‘unobservable’ knowledge insight/rules/discretion which
correctly assign individuals to treatment/control groups. “This person is a 2.1”
UoH 27th April 2016
60
8
6
Percent
4
2
0
40
60
80
100
overall_ave
Density of Overall Average Mark
All
n=3210
UoH 27th April 2016
61
Plan of Talk
1.
Context 1:
Evidence and Policy
Context 2:
Theory and Interpretation
2.
Institutional Arrangements
3.
Data and Methodology
4.
Results
5.
Conclusions and Further Work
UoH 27th April 2016
63
IV (2SLS): log(annual pay) on 2:1 Treatment
ALL (2:1 vs 2:2)
(Same as previous estimation: but excluding cases of
3rds/1sts in borderlines)
Total (centered) SS
Total (uncentered) SS
Residual SS
=
=
=
113.6869721
73125.38036
104.6405362
Number of obs
F( 6,
724)
Prob > F
Centered R2
Uncentered R2
Root MSE
=
=
=
=
=
=
731
10.89
0.0000
0.0796
0.9986
.3783
------------------------------------------------------------------------------|
Robust
lannualpay |
Coef.
Std. Err.
z
P>|z|
[95% Conf. Interval]
--------------+---------------------------------------------------------------Upper Second |
.1453312
.1657892
0.88
0.381
-.1796097
.4702722
overall_ave60 |
.0027592
.0099878
0.28
0.782
-.0168166
.022335
female | -.1942985
.0288996
-6.72
0.000
-.2509408
-.1376563
startyr_e | -.0938844
.0470748
-1.99
0.046
-.1861493
-.0016194
startyr_1 |
.0648903
.0445951
1.46
0.146
-.0225145
.1522952
startyr_3 | -.0514248
.045259
-1.14
0.256
-.1401308
.0372811
_cons |
10.01462
.1180723
84.82
0.000
9.783199
10.24603
-------------------------------------------------------------------------------
So the cases of 1sts in the top borderline was driving an
apparent effect of 2:1s (given a discontinuity around 70).
48=< Overall Average Mark <70
UoH 27th April 2016
67
IV (2SLS): log(annual pay) on 2:1 Treatment
Science (2:1 vs 2:2)
Second Stage regression
Total (centered) SS
Total (uncentered) SS
Residual SS
=
=
=
44.13720296
34613.63128
40.36358091
Number of obs
F( 3,
341)
Prob > F
Centered R2
Uncentered R2
Root MSE
=
=
=
=
=
=
345
9.42
0.0000
0.0855
0.9988
.342
-----------------------------------------------------------------------------|
Robust
lannualpay |
Coef.
Std. Err.
z
P>|z|
[95% Conf. Interval]
-------------+---------------------------------------------------------------Upper Second |
.2742169
.1543015
1.78
0.076
-.0282084
.5766422
overall_ave | -.0067806
.0109202
-0.62
0.535
-.0281838
.0146226
female | -.1356737
.0379965
-3.57
0.000
-.2101455
-.061202
_cons |
10.26969
.5676385
18.09
0.000
9.157143
11.38224
------------------------------------------------------------------------------
For First Stage regression see next slide
48=< Overall Average Mark <70
UoH 27th April 2016
68
IV (2SLS): log(annual pay) on 1st Class Degree Treatment
ALL (1st vs 2:1)
Total (centered) SS
Total (uncentered) SS
Residual SS
=
=
=
165.4780731
106057.3973
138.081668
Number of obs
F( 9, 1037)
Prob > F
Centered R2
Uncentered R2
Root MSE
=
=
=
=
=
=
1047
25.15
0.0000
0.1656
0.9987
.3632
-------------------------------------------------------------------------------|
Robust
lannualpay |
Coef.
Std. Err.
z
P>|z|
[95% Conf. Interval]
---------------+---------------------------------------------------------------first |
.1586433
.0730816
2.17
0.030
.0154059
.3018806
overall_ave70 |
-.018867
.0116393
-1.62
0.105
-.0416795
.0039456
overall_ave70*d|
.0428893
.0189292
2.27
0.023
.0057886
.0799899
overall_ave702 | -.0008714
.0005818
-1.50
0.134
-.0020117
.0002689
female | -.1295679
.0245841
-5.27
0.000
-.1777518
-.081384
sci |
.2427035
.0340108
7.14
0.000
.1760436
.3093634
ssci |
.3220823
.0371857
8.66
0.000
.2491997
.394965
startyr_3 | -.0268086
.0243803
-1.10
0.272
-.0745931
.020976
startyr_4 |
.0312276
.036926
0.85
0.398
-.041146
.1036011
_cons |
9.774553
.0701673
139.30
0.000
9.637027
9.912078
--------------------------------------------------------------------------------
58=< Overall Average Mark
UoH 27th April 2016
70
IV (2SLS): log(annual pay) on 1st Treatment
Science (1st vs 2:1)
Total (centered) SS
Total (uncentered) SS
Residual SS
=
=
=
60.61066027
55388.29596
54.67445708
Number of obs
F( 6,
533)
Prob > F
Centered R2
Uncentered R2
Root MSE
=
=
=
=
=
=
540
10.93
0.0000
0.0979
0.9990
.3182
-------------------------------------------------------------------------------|
Robust
lannualpay |
Coef.
Std. Err.
z
P>|z|
[95% Conf. Interval]
---------------+---------------------------------------------------------------first |
.2181923
.1039298
2.10
0.036
.0144936
.4218909
overall_ave70 |
-.022705
.0135287
-1.68
0.093
-.0492208
.0038107
overall_ave70*d|
.0378093
.0197661
1.91
0.056
-.0009314
.0765501
overall_ave702 | -.0003334
.0005811
-0.57
0.566
-.0014724
.0008056
female |
-.139969
.0302722
-4.62
0.000
-.1993014
-.0806366
startyr_e |
.0488098
.0276974
1.76
0.078
-.0054761
.1030957
_cons |
9.942504
.0871684
114.06
0.000
9.771657
10.11335
--------------------------------------------------------------------------------
Consistent with RD for 60 < Overall Average Mark < 70
58=< Overall Average Mark
UoH 27th April 2016
71
5.
Conclusions and Further Work
(i)
(Fuzzy) RD to estimate causal effect of degree class on earnings
(ii)
Evidence of Signalling in early careers
Upper Second Class Premium
For Science only:
27%
First Class Premium on average: 16%
Strongest for Science: 22%
(iii)
Other findings
Large negative female intercept
Premia for Science and Social Science over Arts/Humanities
(iv)
Further Work/Data
4-digit SOC average occupational earnings
Telephone Responses
More cohorts
3 ½ year follow-up
HMRC link
=> Non-parametric results
UoH 27th April 2016
73