R ESEARCH
Building better performance:
an empirical assessment of the
learning and other impacts of
schools capital investment
PricewaterhouseCoopers
Research Report RR407
Research Report
No 407
Building better performance:
an empirical assessment of the learning and
other impacts of schools capital investment
PricewaterhouseCoopers
The views expressed in this report are the authors' and do not necessarily reflect those of the Department for
Education and Skills.
© Queen’s Printer 2003. Published with the permission of DfES on behalf of the Controller of Her Majesty's Stationery
Office. Applications for reproduction should be made in writing to The Crown Copyright Unit, Her Majesty's
Stationery Office, St Clements House, 2-16 Colegate, Norwich NR3 1BQ.
ISBN 1 84185 942 7
March 2003
Contents
EXECUTIVE SUMMARY..................................................................................... i
I
INTRODUCTION.................................................................................................. 1
II
METHODOLOGICAL APPROACH .................................................................. 5
III
KEY FINDINGS FROM QUALITATIVE ANALYSIS ................................... 15
IV
KEY FINDINGS FROM QUANTITATIVE ANALYSIS ................................ 29
IV
CONCLUSIONS................................................................................................... 45
APPENDICES
A : QUALITATIVE ANALYSIS - ADDITIONAL INFORMATION ...................... A1
B: QUANTITATIVE ANALYSIS - ADDITIONAL INFORMATION.....................B1
Executive Summary
Background to study
The Department for Education and Skills (DfES) is firmly committed to improving the
quality of the schools capital stock. A comprehensive strategy for schools capital
investment was developed and published early in 1999 (see http://www.teachernet.gov.uk),
and this has been underpinned by subsequent announcements of major additional
investment. This reflects an understanding that improving the quality of the schools
capital stock is likely to have an important influence on learning outcomes amongst
pupils.
Schools capital investment and pupil performance
Although arguments about the relationship between schools capital investment and pupil
performance are intuitively appealing, there has, until recently, been relatively little
research evidence to support them in the UK. However, in January 2000 a report entitled
‘Building Performance’ was published in the DfES Research Report series. This
presented the findings from a major study commissioned by the DfES, and undertaken by
PricewaterhouseCoopers (PwC) during 1999. The study provided qualitative evidence
and some quantitative evidence to support the view that a positive and significant
association existed between schools capital investment and pupil performance.
The results presented in the original ‘Building Performance’ report were limited,
however, from a statistical point of view on the basis of data availability. In particular,
the measures of capital investment used in the analysis were aggregate measures,
including all types of capital projects. A priori, it would be reasonable to expect some
types of projects (e.g. repairs to windows and roofs) to have a somewhat weaker direct
impact on pupil performance than other types (e.g. ICT-related investment).
The quantitative research presented in the current report, ‘Building Better Performance’,
develops the original research and, in particular, examines the relationship between
different types of capital spending and pupil performance. The methodological approach
involved collecting quantitative data, for the period 1990-2000, in relation to capital
spending, pupil performance and a range of other factors across all schools in 3 local
authorities in England. The final database contained information on more than 900
schools.
The main focus of the analysis was on (a) whether a differential statistical association
could be identified between the various types of capital spending and pupil performance,
and (b) whether there was any evidence to suggest a causal relationship between any of
the different types of capital spending and pupil performance. The key quantitative
findings from the study are as follows:
• The research provides some additional evidence showing a positive and
statistically significant association between capital investment and pupil
performance;
• The most significant evidence, from a statistical point of view, is in relation to
community primary schools. This is due to a number of issues relating to data
quality and coverage for other types of schools; and
• In terms of the different types of capital investment, the strongest positive findings
are in relation measures of investment which can be related directly to the
i
teaching of the curriculum (e.g. ICT-related capital spending, science blocks etc,
referred to by the DfES as ‘suitability’ investment). This is consistent with
expectations since, a priori, one would expect such investment to have a more
direct impact on performance than other types of investment (e.g. repairs to roofs
and windows, referred to by the DfES as ‘condition’ investment).
The broader impacts of schools capital investment
In recent years there has been considerable interest by policy makers in the UK and
elsewhere in the uses to which school buildings can be put, which go beyond the education
of school pupils (e.g. adult learning, childcare). In terms of debates about schools capital
investment in the UK, these arguments are important in relation to ‘value for money’ and
efficiency considerations.
The qualitative research presented in the current report investigates whether any evidence
can be provided to support the view that the impact of schools, and school buildings in
particular, goes beyond learning outcomes amongst school pupils. The qualitative
research involved, firstly, a design stage, during which a conceptual model was developed
to organise the team’s thinking about the nature of the broader benefits of schools capital
investment, and how they might be categorised. This was done on the basis of discussions
with officials in the DfES, as well as a trawl of existing literature on the broader benefits
of schools, and schools capital investment in particular. The model was then used to
provide a framework for the design of a detailed topic list for use in the in-depth
interviews. Secondly, the qualitative research involved a series of in-depth interviews
with headteachers, other teaching and non-teaching school staff, and a range of broader
stakeholders such as community-based organisations, FE college staff and local
businesses.
The key findings from the qualitative research in relation to the broader uses of schools
are as follows:
•
All of the headteachers interviewed indicated that their school was used, to some
extent, by stakeholders in the wider community. However, the nature and intensity of
school usage varied considerably between schools. In particular, schools which were
located in areas of high economic and social deprivation tended, on average, to be
used more by the wider community. This was partly related to the fact that many of
these areas were relatively under-provisioned, in terms of alternative resources, and
so the school effectively acted as a key public resource within the community. Related
to this, schools tend to be ‘local’, which benefited those from poorer backgrounds,
many of whom would be reliant on paying for public transport to attend alternative
locations;
•
The main demand for school facilities was in terms of specialist facilities (e.g. ICT
suites, early years facilities), auditoria (e.g. for use by drama groups and other local
clubs / societies etc) and sports facilities (e.g. outdoor and indoor pitches, swimming
pool etc);
•
The research provided some good examples of schools entering into mutually
beneficial partnership arrangements with other stakeholders (e.g. the school being
used as a local ‘outreach centre’ by FE colleges or local health authorities). The
evidence suggests that the broader community benefits of the use of school facilities
are enhanced, when they are underpinned by effective inter-agency partnership
arrangements;
•
The evidence suggests that, in relation to the broader benefits, schools capital
investment is likely to be characterised by relatively low levels of deadweight,
displacement, substitution and inequity. The qualitative evidence suggests that this is
particularly the case in relatively deprived areas, and likely to be less so in more
prosperous areas. The following ‘drivers’ underpin these findings:
ii
–
‘Captive’ constituency; all children attend school,
notwithstanding exceptional cases. This means that some of
the broader benefits to school pupils (e.g. health benefits)
have almost universal population coverage; and
–
Accessibility; schools tend to be accessible physically, which
is particularly important within areas of high deprivation /
social exclusion. In addition, since schools can command a
sense of ‘ownership’ amongst most sections of the
community, they tend not to suffer from some of the
psychological barriers, or ‘chill factors’, experienced by
some other forms of provision
Other key findings from the qualitative research are as follows:
•
The qualitative research provided some further evidence in support of the view that
improvements in the physical fabric of school buildings can help to enhance pupil
performance, e.g. ‘suitability’ related projects such as science laboratories, ICT
suites, improving teaching and learning in technology-related subjects, and
‘condition’-related projects such as improvements to roofs and windows improving
teacher and pupil morale; and
•
Capital investment on its own is not necessarily enough and rather, pupil
performance is impacted on by a wide range of contextual factors relating to pupils’
overall ‘learning environment’. There is some qualitative evidence to suggest that
improvements in performance brought about by capital investment are likely to be
limited in schools at either end of the attainment distribution. For example, schools
with relatively poor attainment records were often located in socially deprived areas,
and many pupils in these areas received only limited support from their parents in
relation to their schooling. In such schools, the broader learning environment was
so adverse, that headteachers struggled to see how capital investment, on its own,
could help to improve pupil performance.
iii
I Introduction
Background to study
1.1
The Department for Education and Skills (DfES) is firmly committed to
improving the quality of the schools capital stock. A comprehensive strategy for
schools capital investment was developed and published early in 1999 (see
http://www.teachernet.gov.uk), and this has been underpinned by subsequent
announcements of major additional investment. For example, since 1998 a
number of new funding programmes for schools capital have been established,
including formulaic programmes for Local Education Authorities (LEAs) and
schools, the Schools Access Initiative and Seed Challenge. The total level of
central Government investment available for schools buildings has risen from
approximately £680 million in 1996-07 to £3.7 billion in 2002-03 and will rise to
£7 billion a year by 2005-06, of which £1.2 billion is through the Private Finance
Initiative.
1.2
These developments reflect an understanding that improving the quality of the
schools capital stock is likely to have an important influence on learning
outcomes amongst pupils. For example, the 1999 schools capital strategy
highlighted a number of ways in which capital investment could have a positive
impact on pupil attainment. These included improving teacher and pupil morale,
reducing classroom disruptions and, as one LEA has put it, ensuring that teachers
can concentrate on ‘brains, not drains’ (see http://www.teachernet.gov.uk).
Existing research on the capital – performance relationship
1.3
Although arguments for a positive capital-performance relationship are
intuitively appealing, there has, until recently, been relatively little research
evidence to support them in the UK. However, in January 2000 a report entitled
Building Performance was published in the DfES Research Report series.1 This
presented the findings from a major study commissioned by the DfES, and
undertaken by PricewaterhouseCoopers (PwC) during 1999. The study involved
two main primary research strands as follows:
•
Qualitative research, involving in-depth interviews with headteachers, LEA
officials and other key stakeholders; and
•
Quantitative research, involving a statistical analysis of a dataset containing
information on capital spending, pupil performance and other relevant
factors for nearly 2,000 English schools.
1
Building Performance: an empirical assessment of the relationship between schools capital investment and
pupil performance, report prepared by PricewaterhouseCoopers, DfES Research Report Series, RR242, January
2000.
1
1.4
The findings from this study were broadly consistent with those from similar
studies conducted in the US and elsewhere (which were also reviewed as part of
the Building Performance research). For example, the qualitative evidence
identified a range of factors on which capital investment was judged to have a
strong influence such as teacher morale, pupil motivation and the quality of
teaching, all of which, in turn, were likely to have a significant impact on pupil
performance. Similarly, the balance of quantitative evidence was broadly
consistent with the existence of a positive and statistically significant association
between capital and performance.
1.5
It is important to note, however, that the quantitative part of the assignment also
suggested a certain degree of ambiguity with respect to the estimated statistical
association between capital and performance. In particular, although the balance
of evidence supported a positive and statistically significant association, the
estimated coefficients were not always consistent with this. For example,
sometimes the estimated coefficients, although positive, were not statistically
significant, and sometimes they were not even positive. Again, this apparent
statistical ambiguity is consistent with the findings from similar quantitative
studies which have been conducted in the US and elsewhere.2
1.6
In the Building Performance report, this ambiguity was explained in terms of a
range of factors, one of the most important of which related to data quality. In
particular, as with many of the US studies, the measures of capital investment
used in the analysis were aggregate measures, including all types of capital
projects. These ranged from ‘condition’ investment such as repairs to roofs and
windows, through to ‘suitability’ investment such as the building of ICT suites
and science blocks (see Box below). A priori, it would be reasonable to expect
the former projects to have a somewhat weaker direct impact on pupil
performance than the latter. Unfortunately, however, it was not possible to
investigate this in the original Building Performance research, because the only
information available on capital spending at that time was aggregate-level
information. Disaggregated information on the amount spent on different types
of projects was not available. The quantitative research presented in this report
develops the original Building Performance research and, in particular, examines
the relationship between different types of capital spending and pupil
performance.
Categories of capital spending
Condition – ensuring that the premises are in a good enough state to enable the
children to be educated
Sufficiency – ensuring that there are enough places to educate all the children
Suitability – ensuring that the premises are as appropriate as possible for the
teaching of the curriculum
2
See, for example, Hanushek, E. A. ‘Measuring investment in education’, Journal of Economic Perspectives,
1996, 10, 9-30.
2
Understanding the broader impacts of schools capital
investment
1.7
In recent years there has been considerable interest by policy makers in the UK
and elsewhere in the uses to which school buildings can be put, which go beyond
the education of school pupils. It is argued that the school estate has a key part to
play in terms of reducing social exclusion through, for example, the provision of
childcare facilities, and courses for adults in basic skills or ICT. Such arguments
can be placed in the context of broader policy debates, emanating from the US in
the 1960s, about the ‘community use’ of schools, and the notion of the school as
a key public resource placed at the hub of the local community.3
1.8
For schools capital investment in the UK, these arguments are important in
relation to ‘value for money’ and efficiency considerations. In particular,
securing maximum value for money is one of the key principles upon which the
DfES’s schools capital strategy has been based. In order to measure value for
money, an understanding is required of all the outcomes accruing from the
investment, so that these can be assessed against the overall amount of
investment. In light of this, therefore, it is understandable that DfES has been
keen to explore whether there is evidence to support the view that the impact of
schools, and school buildings in particular, goes beyond learning outcomes
amongst school pupils. The qualitative research presented in this Report explores
the evidence base for this.
Terms of reference
1.9
3
It is within this broader context that the DfES commissioned PwC in January
2001 to develop the original Building Performance research, and to conduct a
further evaluation study of schools capital investment. The study had two main
objectives (see Figure below). The first objective involved ‘going beyond the
school gates’, and examining whether or not the claims made about the potential
broader impacts of schools capital investment, could be provided with an
empirical basis. This objective has been addressed in the qualitative part of this
study. The second objective involved conducting further testing of the statistical
relationship between capital investment and pupil performance, focusing in
particular on the relative impacts of different types of capital spending. This
objective was addressed in the quantitative part of the study. In this regard, the
study was focused on addressing some of the issues which, on the basis of data
limitations, the original Building Performance research was unable to address.
See, for example, 21st Century Community Learning Centers, US Department of Education, 2000.
3
Objectives of study
Relationship between capital
investment and pupil
performance
Broader impacts of schools
capital investment
Objective 1 : To assess whether,
and to what extent, school capital
investment can be shown to have an
impact on broader outcomes, e.g.
health and social outcomes.
Objective 2 : To examine and test
empirically the relationship
between different types of capital
spending and pupil performance.
- focus on range of educational
and other outcomes
- “beyond the school gates”
- educational focus
- “within the school gates”
Structure of report
1.10
This Report sets out the key findings from the research and is structured as
follows:
•
Section II: Methodological approach;
•
Section III: Key findings from qualitative analysis;
•
Section IV: Key findings from quantitative analysis;
•
Section V: Conclusions;
•
Appendix A: Qualitative analysis – additional information; and
•
Appendix B: Quantitative analysis – additional information.
4
II Methodological approach
Overview
2.1
An overview of the methodological approach adopted in the study is provided in
the Figure below. The Figure also shows how each of the two main research
strands, qualitative and quantitative, relate to the two main study objectives.
Overview of our approach
Scoping study
• Selection of stratified sample of 9 LEAs
• Scoping study with these 9 LEAs to assess
(a) availability of quantitative data, (b)
willingness of LEAs to participate in study
• Based on scoping study findings, selection
of 3 LEAs to participate in main study
Qualitative analysis
Quantitative analysis
• Interviews with schools and other
key stakeholders about the broader
impacts of schools capital investment
on:
Learning outcomes
Health/environmental outcomes
Social/community outcomes
Economic outcomes
• Collection of quantitative data on the
nature and extent of capital spending
in each school in 3 LEAs between
1990 and 2001
• Statistical analysis of the relationship
between different types of capital
investment and pupil performance
Objective 1: Broader impacts of
schools capital investment
Objective 2: Relationship between capital
investment and pupil performance
5
Scoping study
2.2
In the scoping study, the study team undertook visits to 9 LEAs throughout
England. LEAs were selected for inclusion on the basis of a number of criteria.
In particular, every effort was made to ensure that an adequate range of different
LEAs were included in the study. The main purpose of the visits was to ascertain
whether or not the existing data held by the LEAs on their capital investment
programmes would be sufficient in their scope and quality to enable a
quantitative analysis to be conducted on the capital–performance relationship. In
relation to this, the requirement was that quantitative data should be available on
a school-by-school basis back to the early 1990s, and relate both to the amount of
capital spending, and the nature of the capital investment projects.
2.3
It was clear from the scoping study that, although records in relation to some of
the more recent capital programmes tended to be reasonably complete, there was
a much greater degree of variability in LEA records with respect to the historical
information (i.e. the information going back as far as 1990). Some LEAs had
held such records in hard copy format, but saw no need to retain the information
relating to, for example, the early or mid 1990s, and eventually the information
was destroyed. Nevertheless, there were a number of LEAs for which historical,
school-by-school data on the nature and extent of capital spending were
available, and three of these were selected for inclusion in the main study.
Therefore, the LEAs selected were chosen largely on their ability to participate in
the study. As such whilst the three areas are reasonably diverse they are not
representative of the types of LEAs throughout the UK. Some key characteristics
of the three LEAs selected for the main study, along with a profile of the schools
in each LEA, are shown in the Tables below.
2.4
It is also important to note at this stage that the sample size for the secondary
school dataset is significantly smaller than for primary schools. This is simply a
direct reflection of the number of secondary schools in each of the LEAs
included in the study. This is important because, as is made clear in Section IV
of the Report, the statistical robustness of the quantitative analysis for primary
schools is significantly better than for secondary schools, particularly when
broken down according to different types of schools [i.e. Voluntary Aided (VA)
and community].4
Profile of LEAs selected for inclusion in study
Size
Urban/
rural
Location
Social
deprivation
Ethnic
minority
share
School
performance
LEA 1
Large
Mixed
North
Med – Low
Med - High
Med – High
LEA 2
Small
Urban
North
High
Low
Med – High
LEA 3
Small
Urban
London
Med - High
High
Med
4
It was beyond the scope of the present study to enhance significantly the secondary school database.
Nevertheless, this issue has highlighted that if a more robust statistical analysis/evaluation is to be conducted in
the future, a more systematic collection of capital-related information on school buildings is needed.
6
Profile of schools in the three LEAs
School
Primary
Of which:
Voluntary aided
Community
Secondary
Of which:
Voluntary aided
Community
Independent
Special
Other
All
LEA 1
479
LEA 2
101
LEA 3
120
All
700
291
188
88
32
69
24
13
107
13
336
364
125
26
62
20
29
3
619
3
21
9
6
140
2
11
7
4
144
31
94
20
45
13
903
Note: Figures include all schools open in the LEA at any time during the period 1990-01.
Approach to qualitative analysis
2.5
The qualitative analysis consisted of two main stages, namely a design /
conceptual development stage, and a fieldwork stage involving a series of indepth interviews (see Figure below).
Overview of approach to qualitative analysis
Stage 1: Design/conceptual development
• Development of conceptual model outlining the various ways in which,
•
in principle, capital investment might have an influence on broader
economic and social outcomes
Development of topic lists for in-depth interviews
Stage 2: In-depth interviews
•
Interviews with schools and other key stakeholders about the broader
impacts of capital investment:
school interviews included headteachers and other teaching and
non-teaching staff
other interviews included, for example, local community-based
organisations, FE colleges, and local businesses
7
2.6
In the design / conceptual development stage of the work, a conceptual model
was developed which tried to organise the study team’s thinking about the nature
of the broader benefits of schools capital investment, and how they might be
categorised. This was done on the basis of discussions with officials in the DfES,
as well as a trawl of existing literature on the broader benefits of schools, and
schools capital investment in particular. The model was then used to provide a
framework for the design of a detailed topic list for use in the in-depth interviews
(the topic list has been reproduced at Appendix A).
2.7
The fieldwork involved a series of in-depth interviews with headteachers, other
teaching and non-teaching school staff, and a range of broader stakeholders such
as community-based organisations, FE college staff and local businesses. The
number of interviews of each type is presented in the Table below.
Number and type of interviews conducted in
qualitative case studies
LEA 1
5
LEA 2
3
LEA 3
4
All areas
12
2
2
1
5
• Users of school facilities (e.g.
community groups, drama clubs)
2
1
1
4
• Alternative providers (e.g. FE
colleges, community-based
organisations)
3
2
2
7
• Local businesses (e.g. architects
and building contractors)
2
1
1
4
All
14
9
9
32
Headteachers/school staff
LEA officials
Other stakeholders
Note: Figures in relation to headteachers/teaching staff refer to the number of schools visited in the
research. On most occasions the visits included interviews with the headteacher and a number of
other teaching and non-teaching staff.
2.8
Every effort was made in the research to ensure that the sample of schools
selected for the qualitative analysis was ‘representative’, in the sense that it
included a range of different types of schools. In order to do this, three broad sets
of criteria were used to select the schools for inclusion in the analysis, namely (a)
characteristics of the local area, (b) characteristics of the school, and (c) key
features of the schools’ recent capital expenditure profile (see Figure below).
The key characteristics of each of the schools, under each of these three headings,
are set out in Appendix A.
8
Criteria for selecting schools for inclusion in
qualitative case studies
•
Urban/rural: schools from both urban
and rural areas selected.
Local area profile
•Socio-economic circumstances: schools
selected from areas of high and low socioeconomic deprivation.
•
Stage: both primary and secondary
schools included in analysis.
School profile
•
Type: Voluntary Aided and community
included in analysis.
•Category: a range of levels of condition,
Capital profile
sufficiency and suitability spending.
Approach to quantitative analysis
2.9
The quantitative analysis consisted of two main stages, namely data collection,
and database development and data analysis (see Figure below).
9
Overview of approach to quantitative analysis
Stage 1 : Data collection
•
•
for individual
funding streams
Collation of paper and electronic records, data input and crosschecking
‘Sign-off’ meetings
– checking of individual records
– discussion of data constraints
Stage 2 : Database development and data analysis
•
•
•
•
2.10
Development of electronic schools capital database
Categorisation of individual capital projects under 3 broad headings of
condition, sufficiency and suitability
Input of data on pupil performance and other variables provided by DfES
and Ofsted
Statistical analysis
– descriptive analysis
– correlation analysis
– multivariate analysis
On-going contact between study team and LEA officials
(e-mail, telephone, fax)
• Introductory letter from PwC to LEA
• Initial set-up meetings with LEA Steering Group
• Bilateral meetings with LEA officials responsible
The data collection exercise was an extremely resource intensive part of the
assignment. It involved a series of meetings with LEA officials, and a
painstaking trawl of existing LEA capital records, held either electronically or in
hard copy format. In many cases, different LEA officials were responsible for
administering and collecting data in relation to the different capital programmes.
Information on capital spending was gathered for a wide range of separate
funding streams / capital programmes. These programmes, along with an
indication of the number of individual schools for which school-specific capital
information was gathered in relation to each programme, is shown in the Table
below.
10
Funding streams from which schools capital
information was drawn
Number of schools with
information available
Major capital works
77
Pre-planned maintenance (PPM)
640
Schools Access Initiative
239
Seed Challenge
96
New Deal for Schools (Phase 1-4)
371
Class Size Initiative
170
Surplus Place Removal
15
Private Finance Initiative (PFI)
9
Excellence in Cities
2
Capital Challenge
4
Energy Efficiency
6
Staff Room Improvement
135
Information and Communication Technologies (ICT)
123
Note: Table shows the number of schools for which information on the nature
and extent of capital spending was drawn from each of the individual funding streams. The names of
the programmes are those given by the LEA records and may not relate directly to official definitions.
2.11
A range of issues arose during the data collection process, relating to the quality
and coverage of the capital data in relation to each of these programmes. In
particular, there were difficulties obtaining accurate and comprehensive capital
information in relation to:
•
VA schools; this problem was also encountered in the original Building
Performance research. The view of the study team was that, on balance, the
quality of the capital data collected by LEAs in relation to VA schools was
generally lower than in relation to community schools and, for this reason,
the quantitative analysis (Section IV) tends to focus more on the results in
relation to community schools. The data quality was poorer as no consistent
approach existed across LEAs for administering/monitoring capital in VA
schools;
•
Recent devolved funding arrangements; often LEAs were unclear about
the profile of spending which, under the recent funding arrangements, had
been devolved to schools. Although this is an important issue more
generally, it posed less of a problem for the quantitative analysis, most of
which was focused on the effects of capital spending in the early and mid1990s, prior to the devolved funding being made available to schools; and
•
Historical spending relating to the early-to-mid 1990s; it was not
surprising that it was more difficult to get quality data for the earlier years
(mainly pre-1994). This was particularly the case for one of the three LEAs
included in the analysis. This LEA was the smallest of the three and, as
such, the problems obtaining historical data had a relatively minor impact on
the overall analysis. Historically the annual summary capital programme
document provided a fairly comprehensive list of school-by-school
investments. This is no longer the case with a wide range of new
programmes supplementing the summary capital programme.
11
2.12
As outlined above, there were just over 900 schools across the 3 LEAs. The final
database of ‘valid’ cases which was used in the analysis was made up of 786
schools. These were the schools which had some capital information (including
‘zero’ values) for the period 1990-95, and pupil performance information for
1996-00. The schools excluded were largely schools that were no longer open or
within the LEA’s jurisdiction. New schools which were not in existence in 1996
were also excluded. The profile of these schools, along with an indication of the
number of schools with condition, sufficiency and suitability-related spending
between 1990 and 1995, is shown in the Table below.
Profile of ‘valid’ cases in database
School type
Total
number
of cases
Number
of ‘valid’
cases
Number of ‘valid’ cases with capital
expenditure during the period 19901995
Condition
Suitability
Sufficiency
Primary
Of which
Voluntary aided
Community
700
634
285
51
70
336
364
331
303
109
176
24
27
25
45
Secondary
Of which
Voluntary aided
Community
125
123
82
33
9
31
94
31
92
14
68
7
26
3
6
Other
78
29
10
2
2
Total
903
786
377
86
81
Note: ‘Valid’ cases are defined as schools which had information on capital spending for the period
1990-95, along with information on performance for the period 1996-00.
2.13
The statistical analysis in Section IV excludes ‘other’ schools, which includes
independent, special and private schools, for both sample size and data
availability reasons.
2.14
The database development and data analysis stage involved, firstly, coding
and classifying the capital data according to the DfES’s three standard capital
categories of condition, sufficiency and suitability. This was completed in
consultation with officials in the DfES. An overview of how individual broad
types of capital projects were classified under the three headings of condition,
sufficiency and suitability is provided in the Table below. Secondly, these data
were then ‘matched’ electronically to other data, provided either by the DfES or
Ofsted, relating to pupil performance and other relevant variables (e.g. school
size, school type, indicators of local economic conditions, teacher motivation
etc). The analysis was based, in total, on 29 separate measures of pupil
performance, 9 for Key Stage 1, 6 for Key Stage 2, 9 for Key Stage 3, 4 for
GCSEs and 1 for A level. A complete list of the individual performance
measures is provided in Appendix B). This ‘disaggregated’ approach to the
analysis of pupil performance was judged to be the preferred approach, since the
analysis was based on a similarly ‘disaggregated’ approach to the measures of
capital investment.
12
Broad classification of capital spending
Type of spend
Classification
Access
Adaptation
Additional
Building alterations / improvements /
new schools
Curriculum works
Consolidation / amalgamation
Create Room
Demolition
Energy Conservation / Health and Safety
Extension / basic need / class reduction
Major suitability capital works
Minor works
Nursery
Playgrounds
Refurbishment
Reorganisation / rationalisation
Repair / replace
Replacing temporary accommodation
Security
Site modifications
Special needs
Staff room
Toilets / removal
Windows
Year end maintenance
TOTAL
Suitability
Suitability
Sufficiency
Suitability
Suitability
Suitability
Suitability
Condition
Suitability
Sufficiency
Suitability
Condition
Suitability
Suitability
Suitability
Sufficiency
Condition
Sufficiency
Condition
Suitability
Suitability
Suitability
Condition
Condition
Condition
-
Number of
instances recorded
83
28
120
119
152
8
152
16
7
260
79
116
24
26
32
43
395
33
468
62
214
135
11
44
585
3,212
Note: Some of the types of spend are not easily classified into the three categories, in particular
consideration was given to classifying Playgrounds and Nursery as Condition, not Suitability. In the
case of playgrounds it was kept as suitability due to the view expressed by teachers that it was often
used as another classroom for teaching and in general it impacted upon the quality of the teaching
environment. Nursery works were also left as suitability as the works encountered were mostly in
relation to improving nursery teaching quality and this was treated in the same manner as quality
improvements at other stages of schooling. It is also worth noting that both categories have less than
30 separate instances of spend and as such, any reclassification would impact only marginally on
overall results.
2.15
The data collected in relation to capital spend were as complete as possible from
the available records. Although it is impossible to be sure how much of the
actual capital budget could be assigned to a particular category, a simple cross
check against published total capital budgets suggests the amount is in the range
70-80% for the earlier years, rising to 90-95% in latter years.
2.16
There are a number of reasons for the inability to allocate out capital spending to
schools. The main reason relates to programmes which were not ‘school
specific’, e.g. fire precautions, nursery programmes. However, in the main these
tended to have more ‘general’ coverage and as such are less likely to impact upon
the results of the study. Another reason for capital not to be allocated is where
the capital is outside the LEA’s control, for example, spending funded privately
by schools. However this is not thought to represent a significant amount of
overall spending and, therefore, is unlikely to impact significantly upon the
results.
13
2.17
In terms of the data analysis, it is in the main based on the nominal capital
information obtained in the data collection exercise. Sensitivity analysis revealed
that deflating the capital data did not impact significantly upon the results given
that the analysis was largely based on capital spending between 1990-955. The
framework used was broadly consistent with the one used in the original Building
Performance research. In particular, a ‘technically pragmatic’ approach was
adopted in which a range of statistical and econometric methods were used to
analyse the data. Generally speaking, these methods fell into one of three types
of analysis, namely descriptive analysis, correlation analysis and multivariate
analysis (see Box below). This approach was adopted in order to ensure that the
results of the analysis did not rely on one particular type of statistical method.
Rather, a balanced view of the overall findings was taken on the basis of the
results drawn from a range of methods.
Overview of approach to statistical analysis
•
Descriptive analysis: providing a statistical overview of the key features of the
data in relation to the two main indicators of capital investment, namely capital
spending (condition, sufficiency and suitability) and the adequacy of
accommodation. This was done mainly using simple descriptive statistics, e.g.
standard deviations, means etc.
•
Correlation analysis: providing a detailed analysis of the correlations which
exist between capital investment and pupil performance. This was done mainly
using simple correlation coefficients and cross-tabulations. Aggregate
correlations were conducted for all the schools on the dataset. In addition,
partial correlations were also conducted, i.e. for certain types of school, or
schools of a certain size etc.
•
Multivariate analysis: investigating the causal relationships between capital
investment and pupil performance. In particular, the multivariate analysis was
used to estimate the impact of capital on performance, whilst simultaneously
‘controlling for’ a range of related factors (e.g. school type, school size etc).
The multivariate models were linear regression models (i.e. Ordinary Least
Squares).
5
The positive statistical associations between capital spending and pupil performance found in the quantitative
analysis also held when deflated capital data was used. Given that deflating past expenditure increases the
numerical value of the capital spend (e.g. £150 in 1995 would equate to £177 at 2000 prices), the strength of the
association (as measured by the regression coefficient) will vary by up to 25 per cent. However the significance
of the association remains broadly the same.
14
III Key findings from
qualitative analysis
Introduction
3.1
The main aim of the qualitative part of this study is to examine whether or not
empirical evidence can be provided to support the view that schools, and schools
capital investment in particular, have a significant impact on a range of outcomes
which go beyond learning outcomes amongst pupils. In order to assess this, a
series of in-depth interviews were conducted with headteachers and other key
stakeholders in the three LEA study areas. This Section outlines the key findings
from the qualitative analysis, and is structured as follows:
•
Towards a simple conceptual model;
•
Schools capital and learning outcomes;
•
Schools capital and social / community outcomes;
•
Schools capital and economic outcomes;
•
Schools capital and environmental / health outcomes;
•
Assessing the net additionality of broader benefits; and
•
Conclusions.
15
Towards a simple conceptual model
3.2
Before conducting the fieldwork, the study team developed a simple conceptual
model of the broader impacts of schools capital investment. This model
identified the main ways in which, potentially, schools capital investment could
have an impact on factors over and above learning outcomes amongst pupils (e.g.
economic or social factors).
3.3
There are three main elements to the conceptual model. Firstly, the model
highlights the importance of distinguishing between the process and the
outcome of capital investment (see Figure below). In particular, the process of
conducting schools capital works can, in itself, have a potentially important
impact on the outcome of the investment. For example, the Kingsdale
demonstration project in south London, funded by the DfES and Southwark LEA,
has proposed a participative approach to capital investment (i.e. one which fully
involves parents, teachers, pupils and other stakeholders).6 It is argued that if
such an approach is adopted, it is likely to enhance the ultimate impact of the
investment on learning and other outcomes.
Distinguishing between the process and the
outcome of capital investment
Process – relating to the process of conducting the
school capital works (e.g. which funding route
was used, the extent of teacher/pupil/parental
influence on design)
Linkage
between
process and
outcome e.g.
nature of
process can
impact on
the extent
and quality
of the
outcome
Schools capital
investment
Cycle of
continuous
improvement
in quality of
capital stock
Outcome – relating to the use of the
existing/upgraded/new facilities
6
Further information on the Kingsdale project can be found at http://www.school-works.org/kingsdale.asp.
16
3.4
3.5
Secondly, the model identifies the four main categories of broader benefits of
schools capital investment (see Figure below):
•
Learning benefits; relating both to school pupils and to adult learning (e.g.
adults using school facilities to study for ICT courses);
•
Economic benefits; both the direct effects of actually conducting the capital
works (e.g. employment of local building contractors), and a range of
indirect effects (e.g. the impact of improved attainment, which might occur
as a result of capital spending, on local economic development, individuals’
lifetime earnings etc);7
•
Environmental / health benefits; relating to pupil health (e.g. removal of
asbestos) and broader environmental outcomes (e.g. the use of schools’
green spaces as community parks); and
•
Social / community benefits; relating to the benefits occurring to the wider
community as a result of the use of school facilities, e.g. the use of schools’
facilities for sporting or other recreational activities, and the impact of
schools on community development through childcare provision, or parental
involvement.
On the basis of these categories, a ‘menu’ of the broader benefits of schools
capital investment was designed, which set out what were likely to be the most
common individual examples of broader benefits. This ‘menu’ provided a range
of specific examples of broader benefits. This, in turn, was used as a ‘checklist’
in the qualitative fieldwork and, related to this, was used to develop a detailed
topic list for the in-depth interviews. Both the ‘menu’ and the topic list are set
out in detail in Appendix A.
Identifying 4 main categories of broader benefits
Learning Benefits
Social /Community Benefits
•
Pupils – e.g. improvement in pupil
motivation and attendance
•
Other learners – e.g. school-based
Adult Basic Learning or ICT courses
•
Recreation & leisure – e.g. use
of school facilities by local sports / arts
clubs
•
Community development – e.g.
school based childcare provision and
enhanced parental involvement /
community ownership
Process
Process
Schools capital
Schools capital
Investment
Investment
Outcome
Outcome
Environmental / Health
Benefits
Economic Benefits
•
Direct – e.g. employment and
income creation amongst local architects
& the associated multiplier effects
•
Indirect – e.g. impact of better
attainment on key drivers of local
economic development, lifetime earnings
etc.
•
Pupil health – e.g. direct impact
of capital works related to health &
safety
•
Environmental – e.g. usage of
schools ‘green spaces’ as community
parks
7
Note also that an additional microeconomic benefit is also likely to accrue to the school in terms of ongoing
capital investment ensuring the longevity of, or ‘protecting’ earlier investments.
17
3.6
Thirdly, an important part of the conceptual model has involved identifying the
evaluation criteria against which the broader benefits should be assessed. This
involved setting out a simple ‘evaluation checklist’ which included standard
indicators of effectiveness and efficiency, such as deadweight, displacement and
substitution, along with other equity-related indicators. Underpinning this
evaluation checklist was the understanding that the qualitative research was ‘in
the spirit’ of an evaluation, not an audit. This meant that, firstly, the research was
focused on trying to identify the net additional impact of schools capital
investment. Secondly, to the extent that it was possible on the basis of qualitative
research, the study involved examining the impact of schools capital investment
over and above the impact which might result from other types of investment
(e.g. investment in community-based projects). In other words, a key focus of
the evaluative element of the assignment involved addressing the question
‘what’s so special about schools?’ (see Figure below).
A simple ‘evaluation checklist’
Key evaluation criteria
Efficiency
• Deadweight – would the outcome have
occurred in the absence of the school
capital investment?
• Displacement – did the school capital
investment displace other activities in
the local community / economy?
• Substitution – did the investment result
in other school-based activities not
happening?
• Cost effectiveness – could the benefits
have been brought about in a more cost
ff i
?
•
•
•
•
Benefits of school capital
Benefits
of school
investment
capital investment
Learning
Economic
Social / community
Environmental / health
“What’s so special about schools?”
18
Equity
• Were the process and outcome benefits
of the school capital investment
accessible to all groups within society
(e.g. age, ethnicity, geographical
location)?
Schools capital and learning outcomes
3.7
In terms of pupils’ learning benefits, the findings from the qualitative research
were broadly consistent with those presented in the original Building
Performance report. In particular, most headteachers viewed capital investment
as having a positive effect on factors such as teacher and pupil motivation and
behaviour, all of which, ultimately, had an important positive influence on pupil
performance. It was interesting that, in line with the results of the original
Building Performance research, even relatively small amounts of capital
expenditure were judged to have an important impact, particularly on teacher
morale.
Voluntary Secondary
“The new classrooms/surroundings stimulated the children who are now well motivated and
eager to learn. Pupils have more respect for the new classrooms than the old facilities. Teacher
motivation has also increased as a result of the capital investment.”
Community Primary
“The refurbishment of the staff-room ‘did wonders’ for staff morale and motivation. While it
may have only represented a token gesture towards addressing the numerous concerns of the
staff, it demonstrated that I took their concerns seriously. Similarly the refurbishment of the
boys toilets improved the behaviour of the pupils in that they respected the new facilities.”
3.8
In terms of the different types of capital investment, expenditure targeted at
developing specialist areas for science, technology and ICT, was generally
considered by most headteachers to have the greatest positive impact on pupil
attainment. In addition, replacing or modifying classrooms whose design
inhibited desirable teaching methods was also reported to have a significant
impact on pupils’ ability to learn.
Voluntary Secondary
“The old classrooms were badly designed and many of the pupils could not see the blackboard
properly. This obviously had a detrimental impact on the pupils’ ability to learn. The new rooms
are more spacious and it is not necessary to cram pupils into every available corner any more.
This has reduced disruption in the class as pupils are no longer asking ‘what that word on the
board is’.”
19
3.9
The evidence suggested that schools can sometimes be more effective than FE
colleges in encouraging participation in adult learning, if the proper facilities are
available e.g. ICT suites.
FE College
“Parents in the local area are poorly educated, intimidated by formal learning and unwilling to
attend the FE college. The school provides a more informal, familiar, and friendly place to
learn. We can use the school to make contact with those we would otherwise be unable to
attract to the campus. In this way we can use the school as a ‘shop window’ for our courses.”
3.10
An interesting finding from the research was that headteachers in schools with
either extremely high or low levels of attainment reported a weaker link between
capital and performance than headteachers in schools with more average
attainment levels. For example, schools with relatively poor attainment records
were often located in socially deprived areas, and many pupils in these areas
received only limited support from their parents in relation to their schooling. In
such schools, the broader learning environment was so adverse, that headteachers
struggled to see how capital investment, on its own, could help to improve pupil
performance. Similarly, in relatively affluent areas, headteachers could not
always see the direct relevance of the capital - performance relationship. Finally,
it is also worth noting that amongst the headteachers interviewed, those in
primary schools tended to report a somewhat weaker link between capital and
pupil motivation, compared to those in secondary schools. This is interesting
given that the quantitative results suggest strongest attainment gains in primary
schools. It may be that the attainment effect in primary schools operates via a
different mechanism to pupil motivation.
Voluntary Grammar
“The tradition and ethos of the school are far more important than the fabric of the
buildings. The poor physical state of our school doesn’t really impact on the
attainment of our pupils, which has always been excellent. The pupils enter the school
with the high level of motivation and desire for learning that comes with an well
educated and professional family background.”
Community Secondary
“The school is in such a deprived area, is so stigmatised, and has such low attainment
levels, that even the complete rebuilding programme that was completed a number of
years ago had little impact on performance. There are so many other social problems,
that the odds are stacked against us. New buildings alone will not address these
problems.”
20
Schools capital and social / community outcomes
3.11
The majority of the headteachers interviewed reported that the schools’ facilities
were used, to some extent, by the local community. This generally entailed a
range of community-based organisations or clubs hiring the school hall and/or the
sports facilities. In addition, some examples were provided of school classrooms
being re-designed and turned into spaces for nursery children. Also, in some
areas included in the research in which there was a large ethnic minority
population, schools often acted as a centre for cultural activities. For example, a
number of headteachers reported the use of their halls for wedding receptions,
and religious festivals. Similarly in some areas with refugee populations, the
local school is often one of the few public places which adults with refugee status
are prepared to visit.
Voluntary Primary
“A classroom was modified four years ago to accommodate a local nursery which
was previously in a very ‘grotty’ scout hut. Once the nursery was moved into the
school, the number of children attending increased dramatically. I would attribute
this to the overall improvement in conditions.”
Leader of local community based organisation
“We wouldn’t be able to meet if we weren’t able to use the school hall. The school
does not charge us and I doubt if we could afford to pay for anywhere else as most
of the girls come from poorer backgrounds.”
3.12
The role of the school, in terms of providing broader social or community
benefits, was generally more important in areas of relatively high economic and
social deprivation. In many such areas the school was often the best publicly
accessible local resource, and in some areas it was essentially the only such
resource. Conversely, people residing in more affluent areas generally had better
access to private transport, and a greater ability to pay for recreation and leisure.
In the absence of school-based activities, such people were more likely to travel
to other facilities outside the immediate vicinity for recreation and leisure.
3.13
This diversity in social/community outcomes relates largely to the schools
‘ethos’. Schools tend to respond to local need. Schools in affluent areas do not
necessarily see their role as providing social and community activities as there is
not ‘additional need’. Conversely, in less affluent areas, where need is higher,
the school responds to this need by providing increased provision.
Voluntary Primary
“The school is very much the hub of the community. There are not many
recreational facilities in the local area, and the school hosts a number of afterschool activities, and sports and drama clubs.”
21
3.14
One FE college included in the research indicated that it ran recreational courses
for elderly people in the local school, e.g. photography, dancing, needlecraft.
The main benefit of using school’s premises was that it was much more
accessible for elderly people than the FE college itself.
FE college
“We organise activities for pensioners which are based in the local high school.
Many of the pensioners we cater for would not be able travel to the College if the
activities were held there. This would impose a travel cost which would in many
cases be unaffordable. It would also place an unacceptable physical strain on many
of the elderly people. In many cases, an afternoon or evening spent making pottery
or flower-arranging represents the only social outlet that these people have.”
3.15
It is important to note that one of the headteachers interviewed indicated that,
shortly after conducting substantial capital works in the school, the school
effectively ‘closed its doors’ to the broader community. This was because the
headteacher was concerned that community-based activities in the schools would
raise awareness of the fact the school now had a lot of new equipment which
would be attractive to local thieves and vandals etc. This suggests that there is a
need for more attention to be paid in the design process to a range of practical
issues, including security.
Voluntary Primary
“I’m sorry to say it, since the work was undertaken I am afraid to open the school up to
the local community.”
22
Schools capital and economic outcomes
3.16
All of the headteachers interviewed stated that they were keen to employ local
firms to conduct capital projects where possible. It is clear, therefore, that all
forms of schools capital expenditure yield direct economic benefits in terms of
income and employment generation among builders / architects, along with the
corresponding multiplier effects. It is important to note, however, that these
benefits are temporary in nature and tend to last only for the duration of the
construction phase. In addition, it was the case that, in relation to larger capital
projects, architects and builders often had to be sourced outside the local area,
representing a significant ‘leakage’ from the local economy.
Property Consultancy Firm
“Where possible, we use local firms who employ local sub-contractors and labour to
carry out the building work. However, on larger projects, it is sometimes necessary to
use big regional or national firms who have the expertise required to conduct the
work.”
Building Firm
“Our firm has been involved in a number of capital works in local schools over the
past few years. Currently we are undertaking a major job replacing window frames
and installing double-glazing in (school name). Large jobs like this provide a
valuable income stream in contrast to some of the smaller contracts which we usually
undertake.”
3.17
A number of examples were provided in the research of suitability expenditure
having longer-term economic benefits in terms of generating on-going
employment for technical support staff (e.g. computer support staff in new ICT
suites). Similarly, sufficiency expenditure can also yield significant longer-term
economic benefits in terms of generating on-going employment for additional
teaching and support staff within the school.8
Voluntary Secondary
“The school has undergone significant expansion over the course of the last four
years with the addition of a new science and technology block and a new art block.
We have had to employ a further fifteen teaching staff, and an additional twenty
support staff as a direct consequence of this investment.”
Community Secondary
“While the expansion programme has led us to increase the number of teaching staff
at the school, none of the new staff hired live locally. However it is quite possible that
some of the labourers employed in the building works are locals. Given the high level
of unemployment in the area, job generation of any kind must be welcome.”
8
It is important to note that, as outlined earlier in this Section and in the ‘menu’ presented in Appendix A, there
is also likely to be a microeconomic benefit accruing from schools capital investment, in terms of the increases in
lifetime earnings for pupils whose performance improved as a result of the investment. Calculating such benefits
requires a formal ‘rate of return’ analysis to be conducted which, although beyond the scope of the current study,
represents an interesting avenue for further research.
23
Schools capital and environmental / health outcomes
3.18
The evidence suggested that schools capital investment could result in a range of
environmental / health benefits. For example, there were direct health benefits
from a range of condition expenditures, e.g. removing asbestos, replacing old and
rotten window frames, replacing heating systems. Such investment can also, in
principle, lead to microeconomic benefits (in terms of savings from energy
efficiency), and also learning benefits (in terms of ensuring that students are
learning within ‘thermally optimal’ conditions). Similarly, spending on school
sports grounds / gymnasiums was judged to have considerable health benefits,
particularly in deprived areas where children had little access to other facilities.
New or improved facilities in the school had the effect of encouraging those who
didn’t participate in sport to do so, and reduced the risk of injury for those who
already did participate. Suitability spending on science laboratories and
technology suites also had safety benefits when old equipment was replaced and
better laid out.
Community Secondary
“The new science labs have greatly enhanced the teachers’ ability to conduct
practical work. The old gas and water mains were not reliable and the layout of the
benches did not allow for group experiments. The new facilities have enabled the
pupils to gain a better understanding of the subject through more ‘hands on’
experience.”
3.19
Schools are often the only ‘green spaces’ in very deprived areas. However, while
acts of vandalism amongst the pupils tended to be rare, the use of school grounds
as community parks had to be limited to prevent others, not involved with the
school, from damaging the property.
Community Secondary
“The investment in the grounds and the nature area in particular has turned the
school into an oasis in a desert of deprivation.”
Community Primary
“The extension of the school building has enabled us to establish a dedicated medical
room. Beforehand, sick children would be sent to the office to wait for a parent
coming to collect them. This obviously disturbed my work and the work of the
secretarial staff, particularly if the child was physically sick. Now children can be
looked after in the medical room as opposed to sitting in the corner of a busy office.”
24
Assessing the net additionality of broader benefits
3.20
As outlined earlier, an important part of the conceptual model was the so-called
‘evaluation checklist’. This involved identifying some of the key questions
which would need to be addressed in order to introduce an evaluation (as
opposed to an audit) element to the assessment of the broader benefits of capital
spending. The main focus of this was on the extent to which schools capital
investment could be associated with key evaluation concepts such as deadweight,
displacement and substitution. And underpinning this was the fundamental
research question, ‘what’s so special about schools?’, i.e. are any of the broader
benefits more likely to come about as a result of investing in schools capital as
opposed to other forms of capital, or other forms of public expenditure more
generally?
3.21
Based on the results of the qualitative research, the Table below provides a
summary of how, in our view, the broader benefits of schools capital investment
‘stack up’ vis-à-vis the four main evaluation concepts. Essentially, the
qualitative research suggests that, in relation to broader benefits, schools capital
investment is likely to be characterised by relatively low levels of deadweight,
displacement, substitution and inequity. The qualitative evidence suggests that
this is particularly the case in relatively deprived areas, and is likely to be less so
in more prosperous areas.
3.22
The following ‘drivers’ underpin these findings:
•
‘Captive’ constituency; all children attend school, notwithstanding
exceptional cases. This means that some of the broader benefits to school
pupils (e.g. health benefits) have almost universal population coverage; and
•
Accessibility; schools tend to be easily accessible physically, which is
particularly important within areas of high deprivation / social exclusion. In
addition, since schools can command a sense of ‘ownership’ amongst most
sections of the community, they tend not to suffer from some of the
psychological barriers, or ‘chill factors’, experienced by some other forms of
provision.
25
An assessment of the ‘evaluation checklist’
Deadweight; would the
outcome have occurred in
the absence of school
capital investment?
Low – Medium
in relatively
deprived areas
Medium – High
in affluent areas
In areas of high economic and social
deprivation the school is often one of
the main public resources available
for use by the community.
In
addition, school facilities are often
available free of charge or at a very
economical
rate
compared
to
alternative facilities.
In more affluent areas, there is a
wider range of alternative provision,
and it is likely that those using school
facilities would have the means to
avail of these.
Displacement; did the
school capital investment
displace other activities in
the local community /
economy?
Low-Medium in
relatively
deprived areas
Medium in
affluent areas
Displacement is less of a problem in
relatively deprived areas because, as
outlined above, such areas tend not to
be well provisioned for in terms of
alternative facilities.
There may be some degree of
displacement in more affluent areas
which may have other existing
facilities.
Substitution; did the
investment result in other
school-based activities not
happening?
Low
Community use of schools generally
takes place outside normal school
hours and, as such tends not to
interfere with day-to-day curriculum
Inequity; were the
benefits of the school
capital investment
accessible to all groups
within society?
Low
Schools tend to score well on equity
criteria. For example, they tend to be
accessible physically, which is
particularly important within areas of
high deprivation / social exclusion.
In addition, since schools can
command a sense of ‘ownership’
amongst most sections of the
community, they tend not to suffer
from some of the psychological
barriers,
or
‘chill
factors’,
experienced by some other forms of
provision.
Note: Illustrative and based on the results of the qualitative research
Conclusions
3.23
This Section has provided some qualitative evidence on the broader benefits of
schools capital investment. The qualitative nature of the evidence means that it
should be treated as illustrative of kinds of broader benefits which are likely to
accrue, their likely strength and the kinds of issues which underpin them. A
number of interesting findings have emerged from the research. In relation to the
links between school capital and pupil performance, the key findings are as
follows:
26
3.24
•
Further qualitative evidence on the capital - performance relationship;
the study has provided further qualitative evidence, based mainly on the
interviews with headteachers, on the links between schools capital
investment and some of the key drivers of pupil performance, in particular,
pupil behaviour / motivation and teacher morale. As such, the results of the
study are broadly consistent with those presented in the original Building
Performance report;
•
The importance of suitability-related spending; the evidence highlights
the fact that, in the view of headteachers, the strongest and most direct
impact on pupil performance is in relation to suitability-related spending
(e.g. science laboratories, ICT or language suites and classroom
modifications). Other types of capital projects tended not to be cited as
having a direct impact on performance although, again in keeping with the
original Building Performance research, some headteachers indicated that
they could have a more indirect impact through intermediate factors such as
motivation, behaviour and morale;
•
Blurring of boundaries between condition, sufficiency and suitability;
notwithstanding the previous point, it was clear that it was difficult for many
headteachers to classify capital spending under one of the DfES’s discrete
headings of condition, sufficiency and suitability. Often, headteachers
suggested, specific projects can contain elements of all three categories. For
example, although in most cases new classrooms were built in order to
accommodate increased pupil numbers (sufficiency), they were generally of
a higher specification than the old classrooms, and included a range of
design features which made it easier for teachers to teach the curriculum
(suitability); and
•
The importance of the ‘learning environment’ context; the evidence
suggests that in many cases capital investment on its own is not enough to
raise standards significantly. Rather, improvements in buildings are likely to
be most effective when they are part of a broader package of measures aimed
at enhancing the overall quality of the ‘learning environment’, physical,
social and intellectual. This was most evident for schools at either end of the
attainment distribution.
In relation to the nature and extent of broader community use of schools, the
key findings from the research are as follows:
•
Widespread variations in the intensity of the community use of schools;
all of the headteachers interviewed indicated that their school was used, to
some extent, by stakeholders in the wider community. However, the nature
and intensity of school usage varied considerably between schools. In
particular, schools which were located in areas of high economic and social
deprivation tended, on average, to be used more by the wider community.
This was partly related to the fact that many of these areas were relatively
under-provisioned, in terms of alternative resources, and so the school
effectively acted as a key public resource at the hub of the community.
Related to this, schools tend to be accessible physically, which benefited
those from poorer backgrounds, many of whom would be reliant on paying
for public transport to attend alternative locations / facilities;
27
•
Narrow range of school facilities used by local community; the evidence
suggests that the main demand for school facilities was in terms of specialist
facilities (e.g. ICT suites, early years facilities), auditoria (e.g. for use by
drama groups and other local clubs / societies etc) and sports facilities (e.g.
outdoor and indoor pitches, swimming pool etc). This suggests that capital
spending in relation to these areas, which tends to come under the suitability
or condition categories, is likely to have the most significant social /
community impact; and
•
The importance of partnership arrangements; the research has provided
some good examples of schools entering into mutually beneficial partnership
arrangements with other stakeholders (e.g. the school being used as a local
‘outreach centre’ by FE colleges or local health authorities). The evidence
suggests that the broader community benefits of the use of school facilities
are enhanced, when they are underpinned by effective inter-agency
partnership arrangements.
28
IV Key findings from
quantitative analysis
Introduction
4.1
The main aim of the quantitative part of this study is to examine the statistical
relationship between schools capital spending and, in particular, different types of
capital spending, and pupil performance. In order to do this, a detailed
quantitative analysis has been conducted of capital data collected from each of
the three LEAs. The overall approach to the quantitative analysis has been set
out in Section II above. The main aim of this Section is to set out the key
findings from the quantitative analysis. The Section is structured as follows:
•
Descriptive analysis;
•
Correlation analysis;
•
Multivariate analysis; and
•
Conclusions.
Descriptive analysis
4.2
This sub-Section describes some of the key features of the capital data which
were collected in the three LEAs included in the study. The Figure below
provides a summary of total capital spending (in real terms) across all three LEAs
between 1990 and 2001. Amongst the key features of the data are the following:
•
When the period is taken as a whole, it is clear that there has been a
significant increase in the total amount of capital spending over the period.
This reflects the fact that, particularly since 1998, a number of new capital
programmes have begun and the amount of capital available has risen
sharply;
•
Suitability spending has seen the largest increase over the decade, reflecting
new schemes targeted specifically at curriculum works and a general
increase in funding levels;
•
Sufficiency spending, though increasing, has remained more static as it is
related more to demographic factors. In the early 1990s, when available
funding was low, meeting the statutory need to provide sufficient pupil
places was the priority use of capital;
29
•
There has been an increase in the overall amount of condition-related
expenditure in recent years. This reflects the fact that some of the new
capital funding programmes, under which significant resources have been
released, have been focused on a range of condition-related projects, mainly
addressing the worst of the condition backlog that had built up; and
•
The increase in overall levels of funding towards the end of the decade also
allowed some suitability needs to be met, though Seed Challenge, introduced
in 2000-2001, was the only programme specifically targeted at suitability,
until the introduction of the formulaic Modernisation programme from
2002-03.
Schools capital expenditure in 3 case study LEAs 1990-2001
£ (million)
90
Total
80
Suitability
70
Condition
60
Sufficiency
50
40
30
20
10
0
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
Note: Tender price index of public sector building, non-housing (PUBSEC), used to deflate nominal
data. The base year for the index is 1995.
4.3
For reasons which were outlined in Section II, the analytical work which has
been conducted has focused on the link between capital spending between 1990
and 1995, and pupil performance between 1996 and 1999. The three LEAs
exhibited no ‘consistent’ difference in spending patterns. As such the group is
amalgamated into one ‘total’ sample throughout the majority of the analysis.
4.4
The Table below provides a summary of the profile of spending between 1990
and 1995 in both primary and secondary schools, in each of the three LEAs.
Summary of school spending
Type of
spend
% schools
receiving no
spend
% schools
receiving
low spend
% schools
receiving
medium
spend
% schools
receiving
high spend
All
schools
Condition
20
20
32
27
100
Sufficiency
68
2
16
14
100
Suitability
52
20
11
16
100
Note:
Low = less than £20k, Medium = between £20k - £100k, High = greater than £100k
Includes all schools (n=903).
Capital spending in nominal terms.
Figures may not sum to 100 due to rounding.
30
4.5
The Table shows that within the six year period of 1990-1995, most (80%) of
schools received some condition spend, almost half received some suitability
spending and less than one third received sufficiency spending. This is consistent
with sufficiency spend addressing areas of local population growth. However
where sufficiency spending occurred it generally amounted to a higher value of
spend, with only 2% of schools getting ‘low’ sufficiency spending. Condition
spending over the same period had a wider range, reflecting the fact that many
schools received many separate small condition spends over the decade.
Suitability historically tended to be larger spends but more recently curriculum
works have added a high number of smaller suitability spends to the analysis.
4.6
The distribution of the Ofsted-based data on the adequacy of the school
accommodation in terms of teaching the curriculum is shown in the Table below.
The patterns are broadly consistent with the patterns displayed by a wider sample
of schools in the original Building Performance report, with the majority of
schools being in the ‘satisfactory’ or ‘good’ categories, and significant numbers,
particularly amongst secondary schools in the second LEA, being in the
‘unsatisfactory’ category. As with the capital data the lack of ‘distinction’
between the three areas adds further weight to the chosen approach of
amalgamating into one single ‘non-area specific’ dataset.9
Adequacy of accommodation gradings
Grade
LEA 1
Very poor
Poor
Unsatisfactory
Satisfactory
Good
Very good
Excellent
LEA 2
Very poor
Poor
Unsatisfactory
Satisfactory
Good
Very good
Excellent
LEA 3
Very poor
Poor
Unsatisfactory
Satisfactory
Good
Very good
Excellent
% of schools assessed
Primary
Secondary
0.2
0.0
2.2
1.5
13.2
14.9
49.1
55.2
28.9
17.9
5.9
10.4
0.4
0
100.0
100.0
0.0
5.3
2.2
5.3
3.3
31.6
47.8
36.8
35.6
10.5
10.0
10.5
1.1
0.0
100.0
100.0
0.0
0.0
5.1
0.0
8.5
20.0
44.1
80.0
33.9
0.0
8.5
0.0
0.0
0.0
100.0
100.0
Note: These data are based on Ofsted inspectors’ reports. On average, there were around 10% of
the schools in the sample, for which this information was not available.
Figures may not sum to 100 due to rounding.
9
Note that these Ofsted figures are based on Ofsted inspectors’ assessments of the adequacy of the school
accommodation for the teaching of the curriculum. All inspectors are provided with clear guidance about the
criteria to be applied when making their judgements. Nevertheless, when used for the present statistical
purposes, it is important to understand that these data, when presented in aggregate form, might lack a degree of
internal consistency, since they are based on the qualitative assessments of a large number of individual Ofsted
inspectors. Further information on the definition of terms, and the guidance given to Ofsted inspectors, can be
found on the Ofsted website (www.Ofsted.gov.uk).
31
Correlation analysis
4.7
This sub-Section describes the key findings from the analysis of the patterns of
correlation which exist between the measures of capital investment on the one
hand, and the measures of pupil performance on the other hand. As outlined
above, the analysis used four separate measures of capital investment, namely
spending on condition, sufficiency and suitability between 1990 and 1995, and
the adequacy of the schools’ accommodation, as assessed in the Ofsted
inspectors’ reports. Each of these measures was correlated against each of the
individual measures of pupil performance, of which there were 15 for primary
schools and 12 for secondary schools.10 The performance measures related to
changes in performance between 1996 and 1999. The patterns of correlation for
the pooled sample of primary and secondary schools are shown in the Table
below.
Bivariate correlation coefficients – primary and secondary schools
Performance measure
Coefficient
Condition
Sufficiency
Suitability
Adequacy
Primary
Key Stage 1 level 2+ in Reading
+
+
Key Stage 1 level 2+ in Writing
+
+
+
Key Stage 1 level 2+ in Maths
+
+
+
+**
Key Stage 1 level 2b+ in Reading
+**
+
+
+
Key Stage 1 level 2b+ in Writing
+
+
+
+
Key Stage 1 level 2b+ in Maths
+
+***
+**
Key Stage 1 Average Points Score in Reading
+*
+
+
Key Stage 1 Average Points Score in Writing
+
+
+
Key Stage 1 Average Points Score in Maths
+
+
+*
+**
Key Stage 2 level 4+ in English
+
+
+
+*
Key Stage 2 level 4+ in Maths
+
+
+
Key Stage 2 level 4+ in Science
+
+
+*
+
Key Stage 2 Average Points Score in English
+
+
+
+*
Key Stage 2 Average Points Score in Maths
+
+
+
Key Stage 2 Average Points Score in Science
+
+**
+**
Summary:
Positive: total
12
10
14
15
Positive: significant
2
0
4
6
Negative: total
3
5
1
0
Negative: significant
0
0
0
0
Secondary
Key Stage 3 level 5+ in English
+
+**
Key Stage 3 level 6+ in English
+
+***
Key Stage 3 level 5+ in Maths
+
+
+
Key Stage 3 level 6+ in Maths
+***
+
+
Key Stage 3 level 5+ in Science
-**
-**
Key Stage 3 level 6+ in Science
+**
+
Key Stage 3 Average Points Score in English
Key Stage 3 Average Points Score in Maths
+***
+**
-*
Key Stage 3 Average Points Score in Science
+***
+***
GCSE Average Points Score
+
+
GCSE 5+ A-C
-**
+
GCSE 5+ A-G
-***
+
Summary:
Positive: total
7
8
3
2
Positive: significant
3
3
2
0
Negative: total
5
4
9
10
Negative: significant
3
0
1
1
Note: A ‘+’ indicates that the correlation between the measure of capital investment and the measure
of performance is positive, i.e. the larger the amount of capital investment between 1990 and 1995,
the greater is the increase in performance between 1996 and 1999.
* coefficient statistically significant at a 10% level.
** coefficient statistically significant at a 5% level.
*** coefficient statistically significant at a 1% level.
Based on a sample of 634 and 123 primary and secondary schools respectively.
10
A level performance measures were excluded on the basis of a sample size and a lack of consistent time series
data.
32
Bivariate correlation coefficients based on nominal capital data.
4.8
In relation to the figures presented for primary schools, a number of points are
worth making:
•
The number of positive correlation coefficients far outweighs the number of
negative ones. For example, of the 60 individual coefficients estimated in
relation to primary schools, 51 had a positive sign and 9 had a negative sign;
•
12 out of the 51 positive coefficients which were estimated were statistically
significant at least at a 10% level;11
•
10 out of 12 of these positive and statistically significant coefficients were in
relation to the suitability-related indicators of capital investment (i.e. either
the amount of suitability-related spending, or the adequacy of
accommodation); and
•
The most significant relationships which were estimated tended to be spread
across the different performance measure subjects (i.e. English, reading,
maths and science).
4.9
Generally speaking, therefore, these findings for primary schools provide some
evidence of a positive association between schools capital and pupil
performance.12
4.10
In relation to the figures presented for secondary schools, a number of points
are worth making:
4.11
•
Generally speaking, the figures presented for secondary schools are more
‘mixed’ with respect to the capital – performance association though some
positive correlation is noticeable in the sufficiency measures with 8 positive
correlations, 3 of which are statistically significant; and
•
Of the 48 individual coefficients estimated, there were 8 statistically
significant positive coefficients, and 5 statistically significant negative
coefficients.
On balance, therefore, these figures provide some quite strong evidence for
primary schools to support the existence of a positive association between capital
investment and pupil performance, but a much more ‘mixed bag’ of evidence in
relation to secondary schools. In this regard it is worth noting that, as outlined in
Section II, because there are fewer secondary schools, the sample size for
secondary schools is much smaller than for primary schools (statistically ‘valid’
overall sample size of 123 compared to 634). Indeed, when the overall secondary
school sample is further broken down by the number of schools receiving
condition, sufficiency and suitability-related spending, the number of
observations in particular categories is so small as to raise questions about the
statistical validity of the results13. Given this, it would seem more sensible to
place a greater weight on the correlation results in relation to primary schools as
opposed to secondary schools. A larger, more reliable, secondary school sample
may have yielded different results.
11
The term ‘statistical significance at a 10% level’ means that we can be 90% sure that there is at least some
correlation between the two relevant variables.
12
Note that, in this regard, suitability-related spending and the adequacy of accommodation are both treated as
indicators of suitability-related investment.
13
Notwithstanding this, it is important to note that the statistical significance of the sufficiency measures, which
is not found in the analysis for primary schools, is at least noteworthy and warrants further investigation.
33
4.12
With this in mind, it is interesting to break down the primary school sample into
community schools and VA schools. As discussed in Section II, the study team
had some concerns about the quality of the capital data in relation to VA schools,
held by LEAs, and, conversely, were much more confident that the data in
relation to community schools were accurate, robust and comprehensive in their
coverage. The Table below presents the patterns of correlation amongst primary
schools separately for VA schools and community schools. It is clear that the
patterns of correlation are particularly strong in relation to community schools,
and particularly for suitability-related measures of capital investment. This is
illustrated in the Figure below which shows that, firstly, of the 60 correlation
coefficients estimated for community primary schools, 45 had a positive sign and
15 had a negative sign and, secondly, all 10 of the statistically significant
coefficients had a positive sign, and none had a negative sign.14
S um m ary of sign and statistical significance of
correlation coefficients - C om m unity P rim ary schools
N u m b e r o f co rre la tio n co e fficie n ts
40
35
30
25
20
15
10
5
0
P o sitiv e
N egativ e
S ta tistica lly
sig n ifica n t
P o sitiv e
N egativ e
S ta tistica lly
in sig n ifica n t
14
It is important to stress that, as indicated in Section II of the report, the data collected by LEAs in relation to
VA schools were generally of a somewhat lesser scope and quality than the data collected by LEAs for
community schools. It is likely that this is one of the main factors underpinning the overall difference in
statistical significance between the coefficients for VA schools and those for community schools.
34
Bivariate correlation coefficients – primary schools only
Performance measure
Coefficient
Community
Key Stage 1 level 2+ in Reading
Key Stage 1 level 2+ in Writing
Key Stage 1 level 2+ in Maths
Key Stage 1 level 2b+ in Reading
Key Stage 1 level 2b+ in Writing
Key Stage 1 level 2b+ in Maths
Key Stage 1 Average Points Score in
Reading
Key Stage 1 Average Points Score in
Writing
Key Stage 1 Average Points Score in Maths
Key Stage 2 level 4+ in English
Key Stage 2 level 4+ in Maths
Key Stage 2 level 4+ in Science
Key Stage 2 Average Points Score in English
Key Stage 2 Average Points Score in Maths
Key Stage 2 Average Points Score in
Science
Summary:
Positive: total
Positive: significant
Negative: total
Negative: significant
Conditional
Sufficiency
Suitability
Adequacy
+
+
+
+**
+
+
+
+
-
+
+
+***
+
+
+***
+**
+
+
+*
+
+
+**
+
+
-
+***
+
-
+
+
+
+
+
+
+***
+
+
+
+
+
+*
+***
+
+
+
+
+
+
7
1
8
0
8
0
7
0
15
6
0
0
15
3
0
0
Voluntary aided
Key Stage 1 level 2+ in Reading
+
+
Key Stage 1 level 2+ in Writing
+
Key Stage 1 level 2+ in Maths
+
+
Key Stage 1 level 2b+ in Reading
+
Key Stage 1 level 2b+ in Writing
+
+
Key Stage 1 level 2b+ in Maths
+
+
+
Key Stage 1 Average Points Score in
+
+
Reading
Key Stage 1 Average Points Score in
+
-*
Writing
Key Stage 1 Average Points Score in Maths
+
+
+
Key Stage 2 level 4+ in English
+
+
+*
Key Stage 2 level 4+ in Maths
+
+**
+
Key Stage 2 level 4+ in Science
+
+
+
+
Key Stage 2 Average Points Score in English
+**
+**
Key Stage 2 Average Points Score in Maths
+
+
+
+
Key Stage 2 Average Points Score in
+
+
+
Science
Summary:
Positive: total
9
9
5
13
Positive: significant
1
0
1
2
Negative: total
6
6
10
2
Negative: significant
0
0
1
0
Note: A ‘+’ indicates that the correlation between the measure of capital investment and the measure
of performance is positive, i.e. the larger the amount of capital investment between 1990 and 1995,
the greater is the increase in performance between 1996 and 1999.
* coefficient statistically significant at a 10% level.
** coefficient statistically significant at a 5% level.
*** coefficient statistically significant at a 1% level.
Based on a sample of 303 and 331 community and voluntary aided primary schools respectively.
Bivariate correlation coefficients based on nominal capital data.
35
4.13
Calculating standard correlation coefficients is a very conventional way of
conducting an analysis of the patterns of correlation between capital and
performance. Another, equally straightforward way, involves examining the
levels and changes in pupil performance amongst those schools which undertook
some form of capital investment, and those which did not. One example of this
kind of analysis is presented in the Figures below relating to performance in
Maths at Key Stage 1 (Appendix B provides a more complete set of calculations,
based on all 15 measures of primary school performance). A number of
interesting findings emerge from this type of analysis:
•
In the example illustrated in these Figures, there was relatively little
difference in the performance levels between those schools which undertook
condition-related projects, and those which did not. In terms of
performance changes, however, it was the case that those schools which
undertook condition-related projects improved their performance by slightly
more than those that did not;
•
In terms of suitability-related projects, performance levels were higher
amongst those schools which did not undertake suitability projects,
identifying possibly that pupils were already benefiting from accommodation
that was more suitable. In terms of performance changes, however, those
schools which undertook suitability-related projects tended to outperform
their counterparts significantly. Of course, the findings presented in these
Figures relate to only one performance measure. However, the figures
presented in Appendix B for all the other measures of primary school
performance suggest that this finding is reasonably robust across the vast
majority of performance measures; and
•
Schools which undertook sufficiency-related projects tended to be the
‘better’ schools, with significantly higher performance levels. This is likely
to reflect the fact that ‘good’ schools tend to attract additional pupils and,
therefore, are more likely to require sufficiency projects. Performance
changes, however, were lower in schools which had undertaken sufficiency
projects, perhaps reflecting the difficulties experienced by schools in terms
of building better performance during periods of expansion.
Average level of Key Stage 1 Performance (Maths) by
type of capital spending
Performance level (1996)
15.3
15.2
Schools with no capital expenditure
15.1
Schools with some capital expenditure
15.0
14.9
14.8
14.7
14.6
14.5
Condition
Suitability
Type of spending (1990-95)
36
Sufficiency
Average change in Key Stage 1 performance (Maths) by
type of capital spending
% change in pupil performance (1996-99)
6
5
Schools with no capital expenditure
Schools with some capital expenditure
4
3
2
1
0
Condition
Suitability
Sufficiency
Type of capital spending (1990-95)
Multivariate analysis
4.14
This sub-Section sets out the key findings from the Ordinary Least Squares
(OLS) multivariate analysis of the relationship between capital investment and
pupil performance. As discussed in Section II, the key focus of the multivariate
analysis is to assess the capital – performance relationship within the broader
context of other factors which are likely to have an impact on pupil performance
and which, at the same time, might be correlated with the measures of capital
investment. The multivariate analysis attempts to ‘control’ for a range of other
such factors (e.g. Free School Meals, school size, teaching quality etc.) in order
to get closer to identifying a ‘causal’ relationship between capital and
performance. In doing so, the success of the analysis will depend on the extent to
which the indicators selected provide a reliable proxy for the factors identified
given the available data and its limitations.
4.15
As with the correlation analysis, the multivariate analysis was conducted in
relation to the 4 standard measures of capital investment and the 27 measures of
performance (15 for primary schools and 12 for secondary schools). The main
purpose of this sub-Section is to set out the findings in relation to the measures of
capital investment. A simple overview of the findings in relation to the other
broader factors which were included in the models is provided in Appendix B.
4.16
The Table below provides a summary of the sign and statistical significance of
the regression coefficients in the multivariate models for primary and secondary
schools. In relation to primary schools, the results are broadly consistent with
the results of the correlation analysis presented in the previous sub-Section:
•
Of the 60 regression coefficients which were estimated, 48 had a positive
sign, indicating a positive relationship between capital investment and pupil
performance;
•
Of these 48 positive regression coefficients, 10 were statistically significant
at least at a 10% level of significance;
•
Most of the statistically significant regression coefficients (8 out of 10) were
in relation to suitability related measures of capital investment; and
•
There were no statistically significant negative regression coefficients.
37
4.17
In relation to secondary schools, the regression results were slightly more
positive compared to the results of the correlation analysis presented in the
previous sub-Section. The key findings are as follows:
•
Of the 48 regression coefficients estimated, 24, exactly one half, had a
positive sign and 24 had a negative sign;
•
Of the 24 regression coefficients with a positive sign, 10 were statistically
significant, and these related to condition and sufficiency indicators of
capital investment; and
•
There were no statistically significant negative regression coefficients.
38
Summary of OLS regression coefficients on capital
variables – primary and secondary schools
Performance measure
Coefficient
Primary
Key Stage 1 level 2+ in Reading
Key Stage 1 level 2+ in Writing
Key Stage 1 level 2+ in Maths
Key Stage 1 level 2b+ in Reading
Key Stage 1 level 2b+ in Writing
Key Stage 1 level 2b+ in Maths
Key Stage 1 Average Points Score in Reading
Key Stage 1 Average Points Score in Writing
Key Stage 1 Average Points Score in Maths
Key Stage 2 level 4+ in English
Key Stage 2 level 4+ in Maths
Key Stage 2 level 4+ in Science
Key Stage 2 Average Points Score in English
Key Stage 2 Average Points Score in Maths
Key Stage 2 Average Points Score in Science
Summary:
Positive: total
Positive: significant
Negative: total
Negative: significant
Condition
Sufficiency
Suitability
Adequacy
+
+
+
+*
+
+
+
+
+
+
+
+
-
+
+
+**
+
+
+
+
+
+
+
+
+***
+
+
+
+
+
+**
+
+
+**
+
+
+*
+
+
+**
+
+
+*
+*
+
+
+
+
+**
12
1
3
0
7
1
8
0
14
3
1
0
15
5
0
0
-
Secondary
Key Stage 3 level 5+ in English
+***
+*
Key Stage 3 level 6+ in English
+**
+***
+
Key Stage 3 level 5+ in Maths
+***
+**
+
Key Stage 3 level 6+ in Maths
+***
+**
+
Key Stage 3 level 5+ in Science
+
Key Stage 3 level 6+ in Science
+
+
Key Stage 3 Average Points Score in English
+**
Key Stage 3 Average Points Score in Maths
+
+**
Key Stage 3 Average Points Score in Science
+
+
GCSE Average Points Score
+
+
GCSE 5+ A-C
+
GCSE 5+ A-G
+
+
Summary:
Positive: total
8
9
2
5
Positive: significant
5
5
0
0
Negative: total
4
3
10
7
Negative: significant
0
0
0
0
Note: A ‘+’ indicates that the correlation between the measure of capital investment and the measure of
performance is positive, i.e. the larger the amount of capital investment between 1990 and 1995, the
greater is the increase in performance between 1996 and 1999.
* coefficient statistically significant at a 10% level.
** coefficient statistically significant at a 5% level.
*** coefficient statistically significant at a 1% level.
Regression equation based on nominal capital data.
39
4.18
Although these results are more positive than the results of the correlation
analysis, it is important to note that, generally speaking, the sample size in
relation to secondary schools was significantly lower than for primary schools
and, for this reason, the regression results for primary schools should be treated
as more robust statistically than those for secondary schools. In this light, it is
interesting to examine the primary school regression results in a little more detail.
The Table below presents the standard regression results separately for VA
primary schools and community primary schools. The results show some
particularly strong patterns of positive correlation for the measures of suitabilityrelated spending (see Figure below). The results in relation to VA schools are
rather less positive although, as outlined earlier in this report, there are some
concerns over the quality of the data in relation to VA schools.
Summary of sign and statistical significance of OLS regression
coefficients - Community Primary schools
Number of regression coefficients
35
30
25
20
15
10
5
0
Positive
Statistically
significant
Negative
40
Positive
Statistically
insignificant
Negative
Summary of OLS regression coefficients on capital
variables – primary schools only
Performance measure
Coefficient
Community primary
Key Stage 1 level 2+ in Reading
Key Stage 1 level 2+ in Writing
Key Stage 1 level 2+ in Maths
Key Stage 1 level 2b+ in Reading
Key Stage 1 level 2b+ in Writing
Key Stage 1 level 2b+ in Maths
Key Stage 1 Average Points Score in Reading
Key Stage 1 Average Points Score in Writing
Key Stage 1 Average Points Score in Maths
Key Stage 2 level 4+ in English
Key Stage 2 level 4+ in Maths
Key Stage 2 level 4+ in Science
Key Stage 2 Average Points Score in English
Key Stage 2 Average Points Score in Maths
Key Stage 2 Average Points Score in Science
Summary:
Positive: total
Positive: significant
Negative: total
Negative: significant
Conditional
Sufficiency
Suitability
Adequacy
+
+
+*
+
+
+
+
-
-***
-*
-*
+
+
+
+
+
+
+**
+**
+
+
+***
+**
+***
+***
+
+
+*
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
7
1
8
0
5
0
10
3
15
7
0
0
14
0
1
0
-
Voluntary aided
Key Stage 1 level 2+ in Reading
+
+
+
Key Stage 1 level 2+ in Writing
+
+
-*
Key Stage 1 level 2+ in Maths
+
+
+
Key Stage 1 level 2b+ in Reading
+
+
Key Stage 1 level 2b+ in Writing
+
Key Stage 1 level 2b+ in Maths
+
+
+
Key Stage 1 Average Points Score in Reading
+
+
Key Stage 1 Average Points Score in Writing
+
-*
Key Stage 1 Average Points Score in Maths
+
+
+
Key Stage 2 level 4+ in English
+
+
+
+*
Key Stage 2 level 4+ in Maths
+
+***
+
+
Key Stage 2 level 4+ in Science
+
+
+
+
Key Stage 2 Average Points Score in English
+
+**
Key Stage 2 Average Points Score in Maths
+
+
+
+
Key Stage 2 Average Points Score in Science
+
+
+
+
Summary:
Positive: total
14
11
5
12
Positive: significant
0
1
0
2
Negative: total
1
4
10
3
Negative: significant
0
0
2
0
Note: A ‘+’ indicates that the correlation between the measure of capital investment and the measure of
performance is positive, i.e. the larger the amount of capital investment between 1990 and 1995, the greater
is the increase in performance between 1996 and 1999.
* coefficient statistically significant at a 10% level.
** coefficient statistically significant at a 5% level.
*** coefficient statistically significant at a 1% level.
Regression equation based on nominal capital data.
41
4.19
The Table below provides the regression equations which were used to generate
the results in relation to a sample of four of the performance measures detailed in
the previous Table (see Appendix B for the specification of the variables used in
the regression models).
Sample OLS regression equations Community primary schools only
Control variables
Background
Equation 2
Key Stage 1 level 2+
in Writing
Key Stage 1
Average Points
Score in Reading
Equation 3
Equation 4
Key Stage 1 Average
Key Stage 1 Average
Points Score n
Points Score in Maths
Writing
t
coeff
t
coeff
t
coeff
t
27.662
1.626
2.328
0.304
2.293
0.297
-6.082
-0.792
Condition
spending
2.648
0.267
3.386
0.756
2.661
0.591
-1.571
-0.35
Suitability
spending
5.925
2.18
2.667
2.177
3.726
3.022
4.308
3.511
Sufficiency
spending
-5.897
-1.596
-2.374
-1.426
-3.291
-1.964
-3.162
-1.896
Adequacy of
accommodation
2.465
1.259
0.954
1.081
0.55
0.619
1.282
1.45
-5.519
-3.305
-1.013
-1.346
-0.776
-1.024
-1.041
-1.382
-0.604 0.006.10
1.058
-0.000621
-0.107
0.00121
0.208
1.803
0.397
-1.862
-0.91
2.499
1.212
0.781
0.381
-0.06.58
-0.12
-1.647
-0.689
0.642
0.267
-1.927
-0.805
Adequacy of
resources
0.835
0.296
-0.776
-0.611
1.575
1.233
1.441
1.133
Teaching
materials
-3.689
-1.222
0.06.9
0.051
-1.417
-1.035
1.074
0.788
Quality of
leadership
0.542
0.353
0.344
0.498
-0.058
-0.083
-0.866
-1.25
Low long-term
unemployment
4.099
0.593
-0.863
-0.277
2.833
0.904
-1.861
-0.597
High long-term
unemployment
-2.596
-0.232
-4.253
-0.845
-0.16
-0.032
-4.848
-0.962
Prior attainment
Number of
pupils
LEA 1
LEA 2
Diagnostics
Equation 1
coeff
Constant
Capital
Dependent variable
N
0.007.73
252
252
252
252
R
0.092
0.056
0.071
0.109
F
1.871
1.1
1.411
2.248
2
Note: Regression equation based on nominal capital data
4.20
The ‘R2’ statistics detailed in the Table above quantifies the extent to which
variation in the dependent variable can be explained by the Control variables. For
instance, the ‘R2’ statistics reveals that, in the case of Equation 1, 9.2 per cent of
the variation in the dependent variable (i.e. performance change in Key Stage 1
Level 2+ in writing) can be explained by variations in the Control variables.
42
4.21
The Table below provides an illustration of the strength of the statistical
relationship between capital spend and pupil performance on the basis of
Equation 1 in the previous table. Of the four equations presented in previous
table, Equation 1 shows the strongest impact of suitability spending on pupil
performance.
Illustration of the strength of the impact of suitability spending
Scenario 1
Scenario 2
Scenario 3
Background information
Amount of suitability spending per pupil during
£ 0 per
£ 150 per
£ 300 per
1990-1995
pupil
pupil
pupil
(1995 prices)
Proportion of pupils achieving KS1 level 2 in
78.1%
78.1%
78.1%
Writing (1996)
Proportion of pupils achieving KS1 level 2 in
84.2%
84.7%
85.2%
Writing (1999)
Proportionate increase in KS1 level 2 in Writing
7.8%
8.5%
9.2%
performance
(1996-99)
Impact on the average primary school
Total number of pupils achieving KS1 level 2 in
253
255
257
Writing (assuming 1999 performance)
Additional number of pupils achieving KS1 level
2
3
2 in Writing (assuming 1999 performance) over
the number under Scenario 1
Impact on all primary schools in the three case study LEAs
Total number of pupils achieving KS1 level 2 in
137,000
137,900
138,700
Writing (assuming 1999 performance)
Additional number of pupils achieving KS1 level
900
1,700
2 in Writing (assuming 1999 performance) over
the number under Scenario 1
Impact on all primary schools in England
Total number of pupils achieving KS1 level 2 in
3,747,000
3,771,000
3,794,000
Writing (assuming1999 performance)
Additional number of pupils achieving KS1 level
24,000
47,000
2 in Writing (assuming 1999 performance) over
the number under Scenario 1
Notes:
1.
Increase in performance based on estimated coefficient on suitability spending per head
of 0.005 (OLS regression results for community primary schools in the 3 case study
LEAs);
2.
Regression example based on nominal capital data. However, to illustrate the real cost of
capital expenditure, £150 of capital expenditure in 1995, would equate to £177 of
expenditure in real terms at 2000 prices [Based on the tender price index of public sector
building, non-housing (PUBSEC). The Base year for the index is 1995]. Also, £300 at
1995 prices, would equate to £353 at 2000 prices;
3.
The average primary school is assumed to have approximately 300 pupils, based on the
sample data used in the OLS regression analysis;
4.
The total number of primary school pupils in the ‘3 case study LEAs’ was assumed to be
approximately 163,000, based on the results of 2001 Annual Schools Census;
5.
The total number of primary school pupils in ‘all LEAs’ was assumed to be
approximately 4,451,000, based on the results of the 2001 Annual Schools Census;
6.
The estimated number of pupils achieving KS1 level 2 in writing is based on the total
school population across six cohorts [i.e. proportion achieving KS1 (assuming 1999
performance) multiplied by total school population (all six cohorts or year groups)] This
reflects the overall impact of capital on all six cohorts passing through the school though
the effect will be felt on many more cohorts depending on the life of the building or
equipment;
7.
For example, under scenario 1 approximately, 137,000 pupils would have achieved KS1
level 2 in Writing in the three LEAs over a 6 year period without any suitability capital
investment. However, under scenario 2, where £150 per pupil is spent on suitability
capital investment across primary schools in the three LEAs, an extra 900 pupils would
have achieved this level over the 6 years in addition to the 137,000 pupils who would
have done so regardless; and
8.
Figures may not sum due to rounding.
43
Conclusions
4.22
This Section has provided some quantitative evidence on the impact of schools capital
investment and, in particular, different types of investment, on pupil performance.
The main aim of the analysis was to test whether or not there was a positive and
statistically significant relationship between schools capital investment and pupil
performance. This was done by constructing a detailed quantitative dataset for the
three case study LEAs, and subjecting the data to a range of statistical tests and
sensitivity analysis. Amongst the key findings to emerge from the research are the
following:
•
The research provides some additional evidence to support the view that
there is a statistically significant association between capital investment and
pupil performance;
•
The strongest evidence, from a statistical point of view, is in relation to
community primary schools. This is due to a number of issues relating to
data quality and coverage for other types of schools;15 and
•
In terms of the different types of capital investment, the strongest positive
findings are in relation to suitability-related measures of investment. This is
consistent with expectations since, a priori, one would expect suitabilityrelated capital investment to have a more direct impact on performance than
condition or sufficiency-related investment.
15
Firstly, the analysis in relation to secondary schools is weaker from a statistical point of view because the
sample size is significantly lower compared to primary schools. Secondly, the analysis in relation to VA schools
is weaker because there were concerns at the fieldwork stage that, on account of the way in which the
information used is collected, the data in relation to VA schools was likely to be less accurate than the data in
relation to community schools.
44
V Conclusions
Summary of key findings from qualitative analysis
5.1
The qualitative part of this assignment examined the impact of schools, and
schools capital investment in particular, on broader factors such as economic,
social and community development. By way of summary, the key findings to
emerge from this part of the study are the following:
•
The qualitative research provided some further evidence in support of the
view that improvements in the physical fabric of school buildings can help to
enhance pupil performance. Evidence was provided of this happening
through a range of routes, e.g. ‘suitability’ related projects such as science
laboratories, ICT suites, improving teaching and learning in technologyrelated subjects and in better classrooms, and ‘condition’ related projects
such as improvements to roofs and windows improving teacher and pupil
morale;
•
It was clear from the qualitative research, however, that capital investment
on its own is not necessarily enough and rather, that pupil performance is
impacted on by a wide range of contextual factors relating to pupils’ overall
‘learning environment’. An interesting finding from the research was that
sometimes the impact of capital investment seems to be inextricably linked
to these broader factors. For example, there is some qualitative evidence to
suggest that improvements in performance brought about by capital
investment are likely to be limited, both in areas which are economically
very deprived, and in areas which are economically very prosperous. In
economically deprived areas, there are so many other contextual factors
which are militating against improvements in pupil performance, that even
very significant enhancements in the quality of school buildings, are likely to
do little on their own to improve performance. In economically prosperous
areas, pupils tend to be highly motivated to learn, largely on account of
positive family attitudes towards their learning and, as such, capital
improvements are, in the views of some headteachers, relatively
inconsequential;
•
It is clear that schools, and school facilities in particular, represent a wider
resource which can be used by the broader community. As such, it is
important for any assessment of the benefits of schools capital investment to
reflect the broader benefits, which go beyond learning outcomes amongst
school pupils. Examples of such benefits which have been highlighted in the
qualitative research include:
– The use of schools’ facilities by local FE colleges to provide training in
ICT to adult learners;
– The recreational use of schools’ facilities (e.g. sports facilities or
auditoria);
– The impact on health outcomes amongst pupils resulting from a range of
health & safety-related capital investment (e.g. removal of asbestos
roofing); and
45
– A range of economic effects including the direct employment of building
contractors and architects.
•
It is clear that schools capital investment is likely to result in a range of
broader benefits. What is less clear, perhaps, is the extent to which, in terms
of a ‘macro’ view of public expenditure, a case can be made for prioritising
schools capital investment over other forms of investment. This is a major
issue which involves the social benefits of higher levels of educational
attainment, and it is beyond the scope of the present study to address it in
full. Nevertheless, the research has provided some interesting findings
which may be used to inform a higher-level debate about public expenditure
priorities. In particular, the research suggests that schools capital investment
is likely to be characterised by relatively low levels of deadweight,
displacement and substitution, particularly in relatively deprived areas.
Summary of key findings from quantitative analysis
5.2
The quantitative part of the assignment examined the impact of schools capital
investment on pupil performance focusing, in particular, on the impact of the
different types of capital investment. By way of summary, the key findings to
emerge from this part of the study are the following:
•
The research provides some additional evidence to support the view that
there is a statistically significant association between capital investment and
pupil performance;
•
The most significant evidence, from a statistical point of view, is in relation
to community primary schools. This is due to a number of issues relating to
data quality and coverage for other types of schools; and
•
In terms of the different types of capital investment, the strongest positive
findings are in relation to suitability-related measures of investment. This is
consistent with expectations since, a priori, one would expect suitabilityrelated capital investment to have a more direct impact on performance than
condition or sufficiency-related investment.
46
Appendix A:
Qualitative analysis –
additional information
• Profile of schools included in qualitative case studies
• A simple ‘menu’ of broader benefits
• Topic list for qualitative interviews
A1
Profile of schools included in qualitative case studies
Local area profile
Urban
LEA 1
School 1
9
LEA 1
School 2
9
LEA 1
School 3
LEA 1
School 4
Rural
High
deprivation
School profile
Low socioeconomic
deprivation
Primary
9
9
9
9
School 5
LEA 2
School 6
9
9
LEA 2
School 7
9
9
LEA 2
School 8
LEA 2
School 9
9
LEA 3
School 10
9
LEA 3
School 11
9
9
LEA 3
School 12
9
9
9
County
Condition
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
A2
9
9
9
9
Suitability
9
9
9
Sufficiency
9
9
9
LEA 1
VA
9
9
9
Secondary
Capital profile
9
9
9
A simple ‘menu’ of broader benefits
Category
Learning
SubMain benefit
Process
category
Pupil motivation / behaviour /
Yes
Pupils
attendance/qualifications
Pupil learning time
Yes
Teacher motivation
Yes
Vocational and other learning
Yes
outcomes
Outcome
Adult Basic Learning (ABL)
No
Yes
Training in ICT and other
specialist subjects
No
Yes
Training and development of
contractors' staff
Yes
No
Schools' income generation
No
Yes
Employment and income
creation
Yes
Yes
Direct business usage of
school premises
Pupils' income generation
No
Yes
Yes
Yes
No
Yes
Other
learners
Economic Direct
Indirect
Key drivers of local economic
development
Yes
Relevant types of capital
spend
Generic
Yes
Yes
Yes
Generic
Generic
Generic
Examples / evidence / comment
Evaluation evidence presented in the original Building Performance report (Phase I)
Phase I evaluation evidence
Phase I evaluation evidence
i.e. better school performance 'feeds in' to better inputs for FE/HE providers. Large
body of evidence showing positive relationship between school inputs and FE/HE
outputs. Note, this is based on the existence of a positive relationship between
capital investment and school performance.
Provision of appropriate
Often this is done in partnership with FE colleges, universities or other training
classroom space for adults.
providers
ICT suites and other specialist Often this is done in partnership with FE colleges, universities or other training
facilities
providers. Note that teachers may benefit directly from this form of specialist
investment (e.g. teachers may use ICT suites), thereby contributing to lifelong
learning objectives.
Generic, but greater impact
i.e. amongst local architects and builders
expected for higher value
added capital projects
Relates specifically to those Many examples can be provided (see DfEE publication, 'Raising standards: opening
parts of the school buildings doors').
which can be rented to
external bodies (e.g.
auditorium, car parks, sports
facilities etc)
Generic, but higher value
Relates mainly to employment / income creation amongst local architects and
added work is likely to have builders, not only with respect to the initial capital outlay, but also the associated
greater impact.
linkage / multiplier effects within the local economy. Note also that some types of
capital investment may lead to some non-teacher job creation within the school (e.g.
administrators / managers for ICT suites, swimming pools etc).
Auditorium; outside
Some examples of local businesses using school facilities to promote products and
premises; classroom space.
facilitate transactions (e.g. car sales and other retail establishments)
Generic
Large body of evidence to show that educational attainment is one of the most
important influences on lifecycle earnings. Note, this is based on the existence of a
positive relationship between capital investment and school performance.
Generic
Key drivers relate to, for example, inward investment, small firm growth,
entrepreneurship etc. Some evidence to show that all such factors are, to varying
degrees, influenced by standards of educational attainment in local areas. Best to
think of this effect in terms of the benefits to 'other learners' (discussed above), and
the impact on vocational and learning outcomes amongst pupils (discussed above).
A3
A simple ‘menu’ of broader benefits (continued)
Category
Social /
community
Environmental /
health
Sub-category
Main benefit
Process Outcome Relevant types of Examples / evidence / comment
capital spend
No
Yes
Indoor and outdoor Many examples can be provided (see DfEE publication, 'Raising standards:
Recreation & Sports clubs (e.g.
tennis, swimming,
sports facilities
opening doors').
leisure
fitness)
Arts clubs (e.g.
No
Yes
Auditorium;
Many examples can be provided (see DfEE publication, 'Raising standards:
drama societies)
classroom space
opening doors').
Parental
Yes
Yes
Generic
Some qualitative evidence exists from Phase I of evaluation, to suggest that
involvement
parental involvement can be enhanced as a result of the process or outcome of
conducting capital works.
Community
Yes
Yes
Generic, although
There is a significant body of US evidence in relation to this (see, for example, the
Community
ownership
some specialist
review by National Clearinghouse for Educational Facilities at
development
facilities / provision http://www.edfacilities.org/ir/community_use.cfm.).
may be particularly
attractive to local
community-based
organisations (e.g.
ICT or ABL)
Childcare
No
Yes
Dedicated childcare Examples can be cited of these being provided on school premises, and adults then
provision
facilities
engaging in other school-based learning activities during 'normal' school hours.
Health and safety
No
Yes
Capital works related e.g. removal of asbestos roofing, playground re-surfacing.
Pupil health
directly to health and
safety requirements
School-based
No
Yes
Generic, although
Relates to direct provision of health services at school (e.g. immunisation
health provision
some scope exists for programmes).
the provision of
specialist healthrelated facilities
of
No
Yes
Generally related to e.g. use of schools' 'green spaces' as community parks.
Environmental Integration
'landscaped'
outdoor works
spaces
Environmentally
No
Yes
Generic
e.g. capital works related to the use of renewable energy sources.
friendly'
efficiency
measures
A4
Topic list for qualitative interviews
Learning benefits of schools capital investment
•
Do you think that recent capital investment in your school has had a direct
influence on pupil attainment?
•
Has recent capital investment had any indirect impact on performance through
‘intermediate’ factors such as pupil behaviour or teacher motivation?
•
What specific types of capital investment have had a particularly strong / weak
direct influence on pupil performance, or indirect influence through factors
such as pupil behaviour or teacher motivation?
•
What links, if any, does your school have with other providers of education
and training in the local area?
•
Does your school provide learning opportunities for learners other than school
pupils (e.g. evening classes or other community-based learning)? If so, could
you please describe these?
•
If these other learners were not involved with your school, do you think they
would training/learning somewhere else. If so, where (e.g. at a local college,
community-based training facility etc.)?
•
Do you think your school can provide a service to other learners which cannot
be provided elsewhere? If so, in what ways do you think the learning
opportunities offered at your school are special?
Economic benefits of schools capital investment
•
Could you please tell us about the capital investment which has been
undertaken in your school during the last 10 years.
– Approximate amount of total investment
– Expenditure on different types of capital investment
– Contact details of architects and local building contractors involved
•
What links does your school have with the local business community? Have
any of your school facilities been used in recent years by the local business
community?
A5
Topic list for qualitative interviews (continued)
•
If so, please describe (note any revenue generating opportunities for schools).
Also, if so, what do you think was particularly useful about using your school
facilities for these purposes, as opposed to some other facility (e.g. local
college or community-based organisation)?
•
Do you think any of your recent capital investment has helped to any create
jobs in the local area (e.g. extent of work undertaken by local architects and
building contractors, managers / administrators of IT suites etc)?
•
Do you think any of the recent capital investment in your school has helped to
stimulate economic development in your local area?
•
If so, in what ways (e.g. providing vocational learning opportunities to other
learners, improving the stock of qualifications in the area and, thereby, helping
to attract investment)?
Social / community benefits of schools capital investment
•
Are your school facilities used in any way by the wider community for
recreation, leisure or other purposes (e.g. sports clubs, drama societies, child
care provision etc)? If so, please describe (explore school’s income earning
opportunities).
•
Do you think the users of your school facilities would be able to find similar
facilities outside of the school (e.g. local leisure centre, theatre, communitybased organisation)? If so, why do you think people choose to use your school,
as opposed to other facilities?
Environmental / health benefits of schools capital investment
•
Have any of your recent capital works been related to health and safety? If so,
do you think these capital works have had a direct positive impact on the health
and safety of pupils?
•
Are your school facilities used by any local health providers? If so, please
describe. What is particularly useful about using your school facilities in this
way?
•
Have any of your recent capital works had a direct environmental impact? If
so, please describe.
A6
Topic list for qualitative interviews (continued)
Interview checklist – for internal use only
Key qualitative research questions
Type of benefit (based on conceptual model)
Learning
Is there a link between the school’s facilities / recent capital
investment in the school, and each of these benefits? If so, please
describe.
What particular types of school facilities / capital investment are
relevant for the links identified above?
Are the benefits accruing from the use of the school’s facilities
additional?, i.e. to what extent could they be obtained using nonschool facilities?; what’s so special about the school
A7
Economic
Social / community
Environmental / health
Appendix B:
Quantitative analysis –
additional information
• Indicators of pupil performance used in the analysis
• Key features of the framework used to weight the capital
expenditure variables in the quantitative analysis
• Total capital spending in each LEA 1990-01
• Average capital spending in each LEA 1990-01
• Number of schools for which capital spending information
was available 1990-01
• Average performance change by type of spending
• Sensitivity analysis – different capital investment measures,
and different weighting procedures
• Specification of variables
• Summary of sign and significance regression coefficients on
controlling variables – primary schools only
B1
Indicators of pupil performance used in the analysis
Primary schools
Key Stage 1
•
•
•
•
•
•
•
•
•
Key Stage 2
•
•
•
•
•
•
Key Stage 1 level 2+ in Reading
Key Stage 1 level 2+ in Writing
Key Stage 1 level 2+ in Maths
Key Stage 1 level 2b+ in Reading
Key Stage 1 level 2b+ in Writing
Key Stage 1 level 2b+ in Maths
Key Stage 1 Average Points Score in Reading
Key Stage 1 Average Points Score in Writing
Key Stage 1 Average Points Score in Maths
Key Stage 2 level 4+ in English
Key Stage 2 level 4+ in Maths
Key Stage 2 level 4+ in Science
Key Stage 2 Average Points Score in English
Key Stage 2 Average Points Score in Maths
Key Stage 2 Average Points Score in Science
Secondary schools
Key Stage 3
GCSE
A Levels
•
•
•
•
•
•
•
•
•
Key Stage 3 level 5+ in English
Key Stage 3 level 6+ in English
Key Stage 3 level 5+ in Maths
Key Stage 3 level 6+ in Maths
Key Stage 3 level 5+ in Science
Key Stage 3 level 6+ in Science
Key Stage 3 Average Points Score in English
Key Stage 3 Average Points Score in Maths
Key Stage 3 Average Points Score in Science
•
•
•
•
•
GCSE 5+ A-C
GCSE 5+ A-G
GCSE 1+ A-G
GCSE Average Points Score
A levels 2+
B2
Key features of the framework used to weight the capital
expenditure variables in the quantitative analysis
An alternative approach to aggregating capital investment over a particular time period is
to weight the capital investment by the amount of time pupils actually benefit from it.
Using this approach, the monetary value of the capital investment in a particular year is
weighted by the number of years that pupils subsequently benefit from it. This facilitates
the comparison of the benefit obtained from a series of capital investments by cohorts of
pupils attending the school at different times. It is important to note that pupils may not
benefit from capital investment in the year that the spending occurs, due to construction
delays and the time taken to adapt teaching practices. The figure below details the weights
applicable to capital expenditure for the different performance measures, while the
subsequent box below provides an example of how the weighted capital variables were
estimated in the analysis, assuming a one-year lag between expenditure and pupil benefit.
Different lag structures were also considered in the sensitivity analysis presented in this
appendix.
KS1
KS2
KS3
GCSE
A ‘LEVEL
Key:
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
3
3
3
3
3
3
3
2
1
0
3
3
3
3
2
1
0
2
1
0
2
1
0
2
1
0
2
1
0
7
7
7
6
5
4
3
6
5
4
3
2
1
0
3
3
3
3
3
3
3
3
3
3
3
2
1
0
5
5
5
5
5
4
3
5
5
4
3
2
1
0
7
7
7
6
5
4
3
6
5
4
3
2
1
0
Time spent completing examination/assessment curriculum
Time spent in school prior to commencing the specific examination/assessment
curriculum
B3
Illustrative example of estimating a weighted capital variable
Background information
Average suitability spending per head in school X of…..
£200 in 1997
£100 in 1996
£150 in 1995
£100 in 1994
1-year time lag before the benefits of capital investment are
gained by the pupils [i.e. pupils benefit in 1999 from capital
investment in 1998].
Therefore
Weighted benefit of capital investment to pupils who attended
the school for three years before completing Key Stage 1 in
1999….
Weighted benefit of capital investment to pupils who attended
the school for three years before completing Key Stage 1 in
1996….
£1,300
[ i.e. £200 (* 2 years)+£100(* 3 years
) + £150(* 3 years) + £100(*3 years)]
£400
[i.e.£ 200(* 1 year) + £100( *2
years)]
For example
The additional benefit of the capital investment received by a
pupil enrolling in School X in 1997 and completing Key Stage
1 in 1999 compared to a pupil who enrolled in 1994 and
completed Key Stage 1 in 1996 would be….
B4
£900
[i.e. £1,300 - £400]
Total capital spending in each LEA 1990-01 (‘000s)
Primary
LEA 1
Secondary
Primary
LEA 2
Secondary
Primary
LEA 3
Secondary
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
Condition
£224
£13
£105
£2,894
£2,119
£2,969
£2,209
£2,117
£4,326
£3,472
£6,004
£9,697
Sufficiency
£804
£2,388
£2,045
£2,643
£3,735
£1,423
£1,819
£1,035
£2,849
£3,394
£5,150
£2,755
Suitability
£2,085
£2,579
£3,115
£1,396
£2,228
£3,279
£1,685
£565
£1,803
£1,536
£1,192
£2,411
Condition
n/a
n/a
n/a
£3,461
£2,120
£1,196
£723
£1,946
£1,702
£1,526
£3,674
£2,211
Sufficiency
£89
£1,318
£712
£48
£200
£223
£800
£4,646
£6,433
£1,583
£2,760
£442
Suitability
£1,538
£4,963
£2,692
£3,180
£3,694
£1,766
£619
£899
£3,407
£1,990
£5,252
£35,161
Condition
n/a
n/a
n/a
£431
£104
£85
£389
£133
£573
£1,248
£30
£349
Sufficiency
n/a
£301
£366
£295
£542
£245
£624
£104
£399
£299
£1,059
£1,271
Suitability
£1,200
£1,025
£1,046
£868
£207
£355
£834
£3,189
£2,143
£462
£1,217
£2,679
Condition
n/a
£88
n/a
£71
£54
£233
£117
£176
£505
£1,073
£1,091
£2,156
Sufficiency
n/a
£140
£620
n/a
n/a
n/a
£285
n/a
n/a
n/a
n/a
n/a
Suitability
n/a
£1,787
£2,019
£2,532
£2,346
£3,712
£5,250
£12,721
£8,981
£7,532
£9,284
£3,000
Condition
£3
n/a
n/a
n/a
£36
£295
£63
£324
£555
£1,880
£1,871
n/a
Sufficiency
n/a
n/a
n/a
n/a
£633
£664
£268
£170
£503
£957
£2,074
n/a
Suitability
n/a
n/a
n/a
n/a
£98
£166
£445
£1,755
£2,071
£4,912
£2,546
n/a
Condition
n/a
n/a
n/a
n/a
£80
n/a
£89
£110
£185
£158
£266
n/a
Sufficiency
n/a
n/a
n/a
n/a
£2,130
£2,257
£126
n/a
n/a
n/a
£198
n/a
Suitability
n/a
n/a
n/a
n/a
£331
£178
£79
£1,324
£545
£525
£434
n/a
Note: n/a indicates that no capital spending was recorded.
B5
Average capital spending in each LEA 1990-01 (‘000s)
1990
Primary
LEA 1
Secondary
Primary
LEA 2
Secondary
Primary
LEA 3
Secondary
Note:
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
Condition
£224
£13
£105
£26
£24
£30
£28
£19
£28
£23
£31
£35
Sufficiency
£115
£217
£120
£147
£144
£71
£91
£54
£203
£87
£82
£64
Suitability
£695
£322
£346
£140
£139
£273
£187
£40
£86
£55
£28
£38
Condition
n/a
n/a
n/a
£93
£71
£39
£38
£75
£50
£35
£61
£37
Sufficiency
£89
£1318
£712
£48
£200
£74
£133
£422
£429
£264
£460
£442
Suitability
£513
£1241
£538
£636
£462
£441
£155
£128
£179
£153
£328
£748
Condition
n/a
n/a
n/a
£8
£2
£1
£6
£2
£7
£16
£30
£7
Sufficiency
n/a
£75
£52
£59
£68
£35
£104
£52
£100
£60
£71
£98
Suitability
£1,200
£342
£174
£87
£34
£89
£208
£1063
£536
£11
£12
£51
Condition
n/a
£44
n/a
£4
£3
£12
£6
£9
£26
£54
£64
£240
Sufficiency
n/a
£140
£620
n/a
n/a
n/a
£142
n/a
n/a
n/a
n/a
n/a
Suitability
n/a
£255
£336
£316
£293
£371
£583
£1817
£998
£685
£774
£130
Condition
£5
n/a
n/a
n/a
£18
£295
£21
£46
£55
£10
£58
n/a
Sufficiency
n/a
n/a
n/a
n/a
£79
£95
£45
£34
£72
£106
£259
n/a
Suitability
n/a
n/a
n/a
n/a
£24
£41
£74
£159
£159
£409
£88
n/a
Condition
n/a
n/a
n/a
n/a
£80
n/a
£44
£27
£62
£32
£44
n/a
Sufficiency
n/a
n/a
n/a
n/a
£710
£752
£42
n/a
n/a
n/a
£198
n/a
Suitability
n/a
n/a
n/a
n/a
£66
£59
£26
£165
£91
£66
£48
n/a
Figures show the average amounts of capital investment in each year, amongst those schools that undertook at least some capital investment in the respective year.
n/a indicates that no capital spending was recorded.
B6
Number of schools for which capital spending information was available 1990-01
1990
Primary
LEA 1
Secondary
Primary
LEA 2
Secondary
Primary
LEA 3
Secondary
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
Condition
1
1
1
114
88
99
80
113
154
152
192
275
Sufficiency
7
11
17
18
26
20
20
19
14
39
63
43
Suitability
3
8
9
10
16
12
9
14
21
28
43
63
Condition
n/a
n/a
n/a
37
30
31
19
26
34
44
60
60
Sufficiency
1
1
1
1
1
3
6
11
15
6
6
1
Suitability
3
4
5
5
8
4
4
7
19
13
16
47
Condition
n/a
n/a
n/a
53
48
61
61
67
78
78
1
50
Sufficiency
n/a
4
7
5
8
7
6
2
4
5
15
13
Suitability
1
3
6
10
6
4
4
3
4
43
101
53
Condition
n/a
2
n/a
19
18
19
19
19
19
20
17
9
Sufficiency
n/a
1
1
n/a
n/a
n/a
2
n/a
n/a
n/a
n/a
n/a
Suitability
n/a
7
6
8
8
10
9
7
9
11
12
23
Condition
1
n/a
n/a
n/a
2
1
3
7
10
18
32
n/a
Sufficiency
n/a
n/a
n/a
n/a
8
7
6
5
7
9
8
n/a
Suitability
n/a
n/a
n/a
n/a
4
4
6
11
13
12
29
n/a
Condition
n/a
n/a
n/a
n/a
1
n/a
2
4
3
5
6
n/a
Sufficiency
n/a
n/a
n/a
n/a
3
3
3
n/a
n/a
n/a
1
n/a
Suitability
n/a
n/a
n/a
n/a
5
3
3
8
6
8
9
n/a
Note: n/a indicates that no capital spending was recorded.
B7
Average performance change by type of spending
Condition
Suitability
Sufficiency
Some
capital
spending
No
capital
spending
Some
capital
spending
No
capital
spending
Some
capital
spending
No
capital
spending
Key Stage 1 level 2+ in Reading
12.48
7.68
6.56
10.84
7.45
11.00
Key Stage 1 level 2+ in Writing
10.99
4.90
6.42
8.62
2.81
9.44
Key Stage 1 level 2+ in Maths
9.02
6.00
10.36
7.49
2.77
8.64
Key Stage 1 level 2b+ in Reading
25.99
14.40
18.25
21.38
18.86
21.33
Key Stage 1 level 2b+ in Writing
58.52
23.90
36.80
44.63
69.10
39.38
Key Stage 1 level 2b+ in Maths
11.98
11.76
35.88
9.53
0.13
14.01
Key Stage 1 Average Points
Score in Reading
3.59
1.87
4.82
2.67
1.48
3.12
Key Stage 1 Average Points
Score in Writing
3.86
1.17
3.68
2.63
1.82
2.89
Key Stage 1 Average Points
Score in Maths
2.13
1.37
5.20
1.47
0.67
2.01
Key Stage 2 level 4+ in English
31.14
46.44
52.62
35.64
31.19
38.17
Key Stage 2 level 4+ in Maths
41.36
52.32
58.47
44.38
43.15
46.04
Key Stage 2 level 4+ in Science
40.57
65.36
65.80
48.20
48.61
49.90
Key Stage 2 Average Points
Score in English
5.94
9.32
9.35
7.06
6.69
7.36
Key Stage 2 Average Points
Score in Maths
5.98
8.73
7.90
6.96
6.90
7.06
Key Stage 2 Average Points
Score in Science
7.43
10.30
10.72
8.28
10.15
8.18
Note: figures show percentage change in pupil performance (1996-00) amongst schools with some capital spending and
schools with no capital spending.
B8
Sensitivity analysis – different capital investment
measures, and different weighting procedures
Capital Expenditure (unweighted)
90 – 93
90 – 94
90 – 95
90 – 96
90 – 97
90 - 98
90 - 99
All correlations
Of which:
Statistical
significant
correlations
Condition
Sufficiency
All correlations
Of which:
Statistical
significant
correlations
All correlations
Of which:
Statistical
significant
correlations
Suitability
Sign of correlation coefficient
+
-
+
-
+
+
+
-
+
-
+
-
+
-
23
4
16
11
19
8
16
11
16
11
14
13
14
13
7
0
3
3
5
3
4
3
4
2
2
3
2
2
12
15
15
12
18
9
18
9
17
10
13
14
14
13
0
0
1
0
3
0
3
0
2
0
1
2
1
2
17
10
18
9
17
10
13
14
18
9
16
11
16
11
7
1
7
1
6
1
3
1
4
1
3
1
3
0
Capital Expenditure (weighted)
1 year lag
2 year lag
3 year lag
4 year lag
5 year lag
-
+
-
+
+
+
-
+
-
14
13
15
12
17
10
21
6
19
8
1
3
3
3
3
3
5
1
4
1
Sufficiency
All correlations
Of which:
Statistical
significant
correlations
All correlations
Of which:
Statistical
significant
correlations
13
14
17
10
17
10
16
11
19
8
2
0
3
0
3
1
3
0
0
0
All correlations
Of which:
Statistical
significant
correlations
13
14
15
12
11
16
13
14
17
10
2
0
2
0
4
0
5
0
6
1
Condition
+
Suitability
Sign of correlation coefficient
B9
Specification of variables
Variable
Specification
Key Stage 1 level 2+ in Writing
Percentage change in pupil performance
(1996-1999)
Percentage change in pupil performance
(1996-1999)
Percentage change in pupil performance
(1996-1999)
Percentage change in pupil performance
(1996-1999)
Spending (£) per pupil (1990-1995)
Spending (£) per pupil (1990-1995)
Spending (£) per pupil (1990-1995)
Ofsted inspectors assessment
Ofsted inspectors assessment
Actual numbers
Dummy variable
Dummy variable
Dummy variable
Ofsted inspectors assessment
Ofsted inspectors assessment
Ofsted inspectors assessment
Dummy variable
Dummy variable
Key Stage 1 Average Points Score in
Reading
Key Stage 1 Average Points Score n
Writing
Key Stage 1 Average Points Score in
Maths
Condition spending
Suitability spending
Sufficiency spending
Adequacy of accommodation
Prior attainment
Number of pupils
Single sex
LEA 1
LEA 2
Adequacy of resources
Teaching materials
Quality of leadership
Low long-term unemployment
High long-term unemployment
B10
Summary of sign and significance of regression
coefficients on controlling variables – primary schools
only
Controlling variable
Positive:
total
Positive:
significant
Negative:
total
Negative:
significant
Prior attainment
0
0
15
5
Number of pupils
3
0
12
0
Single sex
2
0
13
0
Voluntary aided
2
0
13
2
Wirral
6
1
9
0
Ealing
8
5
7
0
Adequacy of resources
7
0
8
2
Teaching materials
4
0
11
2
Quality of leadership
10
3
5
0
Low Long-term unemployment
4
0
11
1
High Long-term unemployment
6
0
9
0
B11
Copies of this publication can be obtained from:
DfES Publications
P.O. Box 5050
Sherwood Park
Annesley
Nottingham
NG15 0DJ
Tel: 0845 60 222 60
Fax: 0845 60 333 60
Minicom: 0845 60 555 60
© Queen’s Printer 2003
Produced by the Department for Education and Skills
ISBN: 1 84185 942 7
Ref No: RR407
www.dfes.go.uk/research