Housing, Regeneration and Planning
The SHARP Study:
Objectives, Design and
Methodology
THE SHARP STUDY:
OBJECTIVES, DESIGN AND METHODOLOGY
Mark Petticrew*, Ade Kearns**, Caroline Hoy***,
Marcia Gibson**** and Phil Mason**
*
London School of Hygiene and Tropical Medicine, formerly of the
MRC Social and Public Health Sciences Unit, Glasgow
**
Department of Urban Studies, University of Glasgow
***
Formerly of the Department of Urban Studies, University of
Glasgow
**** MRC Social & Public Health Sciences Unit, Glasgow
Scottish Government Social Research
2008
This report is available on the Scottish Government Social Research website
only www.scotland.gov.uk/socialresearch.
It should be noted that since this research was commissioned a new Scottish government
has been formed, which means that the report reflects commitments and strategic
objectives conceived under the previous administration. The policies, strategies,
objectives and commitments referred to in this report should not therefore be treated as
current Government policy.
© Crown Copyright 2008
Limited extracts from the text may be produced provided the source
is acknowledged. For more extensive reproduction, please write to
the Chief Researcher at Office of Chief Researcher,
th
4 Floor West Rear, St Andrew's House, Edinburgh EH1 3DG
ACKNOWLEDGEMENTS
The SHARP study was funded by the Scottish Government, the Medical Research
Council, the Chief Scientist Office (part of the Scottish Government Health
Directorate) and the University of Glasgow.
The research was originally
commissioned by Communities Scotland, a Scottish Government executive agency
which was abolished on 1st April 2008, in collaboration with the Chief Scientist
Office.
Our thanks to all the Registered Social Landlords and their housing staff for working
closely with us to facilitate this study. Special thanks to the residents themselves for
agreeing to be interviewed for the study and giving their time on more than one
occasion.
Many individuals deserve our thanks for contributing to the study. Caroline Hoy was
the principal researcher on the study for most of its duration and put a lot of effort into
all phases but especially the initial set up of the study. Catherine Ferrell, Kate
Campbell and Julie Watson of the MRC Social and Public Health Sciences Unit
(SPHSU) organised the fieldwork. Our thanks to the fieldwork team for their efforts
throughout the project and for the assistance of Business Plus for conducting
interviews in the north of Scotland for us. Thanks to Rosey Davidson, Maureen
Hunter and Hilary Young for carrying out interviews for the project. Hilary Thomson
of the MRC SPHSU lent her expertise to the project at various times, particularly in
relation to her own review work on the health effects of housing improvements and in
relation to the qualitative research phases. Elise Whitley joined the team to perform a
lot of the statistical analyses for the final reports, alongside the project’s own
statistician, Phil Mason. Betty Johnstone helped prepare the study reports for
publication.
Thanks to the members of the project Steering Group for their advice, support and
encouragement throughout.
CONTENTS
EXECUTIVE SUMMARY
1
CHAPTER ONE
BACKGROUND
Housing, health and regeneration
Previous research on housing and health
The need for an experimental study
Summary
3
3
5
6
7
CHAPTER TWO
RESEARCH AIMS AND OBJECTIVES
Housing change and health
Housing outcomes
Health outcomes
Neighbourhood change and health
Health improvement in regeneration areas
Sociodemographic and other data
Summary
8
8
9
9
10
11
11
12
CHAPTER THREE
STUDY COMPONENTS
Survey methods
Wave 1: Pre-rehousing questionnaire survey
Wave 2: Post-rehousing postal survey
Wave 3: Follow-up questionnaire survey
Qualitative in-depth interviews
Routine data sources
Summary
13
13
13
13
13
14
14
15
CHAPTER FOUR
SAMPLING AND SURVEY
Scheme selection
Identification of Control Group and matching of households in the 2 study groups
Participant recruitment
Participant retention
Survey management and conduct
Summary
16
16
16
17
17
18
19
CHAPTER FIVE
ACHIEVED SAMPLES
Overall samples
Geographical distribution of the SHARP sample
Housing characteristics of sample
Household characteristics of sample
Respondent demographics
Summary
20
20
20
24
25
27
30
CHAPTER SIX
DATA MANAGEMENT AND STATISTICAL ANALYSES
Data management
Derivation of novel variables
Statistical analyses
Investigating changes in outcomes
Summary
31
31
31
32
34
40
CHAPTER SEVEN
QUALITATIVE RESEARCH
Aims of the qualitative research
Post-Wave 2 qualitative research
Post-Wave 3 qualitative research
Limitations
Summary
41
41
41
43
45
45
CHAPTER EIGHT
STUDY STRENGTHS AND LIMITATIONS
Strengths of the study
Study limitations
Conclusion
Summary
46
46
47
48
49
List of tables
TABLE 5.1:
TABLE 5.2:
TABLE 5.3:
TABLE 5.4:
TABLE 5.5:
TABLE 5.6:
TABLE 5.7:
TABLE 5.8:
TABLE 5.9:
TABLE 6.1:
TABLE 6.2:
TABLE 7.1:
TABLE 7.2:
TABLE 7.3:
DISTRIBUTION OF RESPONDENTS BY UNITARY AUTHORITY, FOR INTERVENTION
AND CONTROL GROUPS AT WAVES 1 AND 3
DISTRIBUTION OF HOUSEHOLDS IN INTERVENTION AND CONTROL SAMPLES BY
SCOTTISH EXECUTIVE URBAN/RURAL CLASSIFICATION (PERCENTAGES)
DISTRIBUTION OF RESPONDENTS BY SOCIAL INCLUSION PARTNERSHIP, FOR
INTERVENTION AND CONTROL GROUPS AT WAVES 1 AND 3
DISTRIBUTION OF RESPONDENTS BY HOUSING TENURE, FOR INTERVENTION AND
CONTROL GROUPS AT WAVES 1 AND 3 (PERCENTAGES)
DISTRIBUTION OF RESPONDENTS BY DWELLING TYPE, FOR INTERVENTION AND
CONTROL GROUPS AT WAVES 1 AND 3 (PERCENTAGES)
DISTRIBUTION OF RESPONDENTS BY HOUSEHOLD TYPE, FOR INTERVENTION AND
CONTROL GROUPS AT WAVES 1 AND 3 (PERCENTAGES)
DISTRIBUTION OF HOUSEHOLD SIZE BY HOUSEHOLD TYPE, FOR INTERVENTION
AND CONTROL GROUPS AND WAVES 1 AND 3 (PERCENTAGES)
SEX OF RESPONDENT BY HOUSEHOLD TYPE, FOR INTERVENTION AND CONTROL
GROUPS (WAVE 1)
MARITAL STATUS OF RESPONDENT BY HOUSEHOLD TYPE, FOR INTERVENTION
AND CONTROL GROUPS AT WAVE 1 (RETROSPECTIVELY)
STUDY GROUP SAMPLES BY SIP STATUS
MEASURE OF CHANGE (WAVE 1 TO WAVE 3) IN SENSE OF COMMUNITY
(INTERVENTION GROUP)
FIRST WAVE QUALITATIVE SAMPLE CHARACTERISTICS
QUALITATIVE SAMPLE TARGET FRAMEWORK
SOCIODEMOGRAPHIC CHARACTERISTICS OF THE SAMPLE
21
22
23
24
25
26
27
28
29
37
39
42
43
44
EXECUTIVE SUMMARY
1.
The Scottish Health, Housing and Regeneration (SHARP) study is a
longitudinal study of the health and social impacts on tenants which result from
moving into new-build socially rented housing. The primary aim was to investigate
the impacts of being rehoused in new-build socially rented property on housing
conditions, neighbourhood outcomes and the health and well-being of tenants. This
report outlines the study objectives, design and methodology.
2.
The public policy interest in reducing health inequalities has resulted in a
widening of the objectives of housing and regeneration programmes to try to deal
with the ‘upstream’ influences on health. These impacts may come about not only as
a result of a change in dwelling, but also through associated changes in landlord and
housing management, and through changes in neighbourhood circumstances.
SHARP represents an attempt to identify the effects of such holistic changes, and to
produce better evidence through a quasi-experimental design which is also multi-site
in nature and thus more generalisable than single site studies.
3.
The methodological aims of the SHARP study were to investigate the effects
of comprehensive housing change (rather than the effects of specific elemental
changes to housing), to use a controlled study design to do so, and to make use of
objective as well as self-reported health data. The main research questions related to
the impacts of rehousing upon residential conditions and community circumstances;
whether these conditions were related to landlord and neighbourhood change as well
as housing change; and whether those people rehoused experienced changes in their
physical health, health behaviours, and mental health and well-being.
4.
Three survey waves (pre-rehousing, post-rehousing, and 2-year follow-up)
were combined with in-depth qualitative interviews at 2 points in time (both after
rehousing) and the collection of secondary health data and landlord tenancy
information.
5.
The sample reflects the distribution of the social housing building programme
in Scotland, with half the sample being in the Greater Glasgow area. Forty-five
registered social landlords (RSLs) participated in the study, involving 57 separate
housing developments. The survey was managed by the Medical Research Council
Social and Public Health Sciences Unit (MRC SPHSU) survey team, with the use of a
contractor to conduct interviews in the more distant locations.
6.
The Intervention Group sample represents around a 10 per cent sample of the
annual national output of generalpurpose social rented housing by RSLs in Scotland at
the time of the study. Three-quarters of the samples were in urban areas (including a
third in the City of Glasgow), and 44-51% were in regeneration areas. Matching of
the 2 study groups by location, household type and tenure worked well: for example,
91% of the Intervention Group and 99% of the Control Group were renting at Wave
1; 72% of the Intervention Group and 63% of the Control Group were families,
respectively.
1
7.
Quality control checks were made on 5% of the survey forms, and in addition
variable range checks and logical checks were made on the entire dataset. Analysis
involved use of both the cross-sectional samples and the longitudinal sample. As
regards the study of outcomes (residential, social, mental health and well-being, and
physical health), a series of investigations was made, looking at: changes over time
comparing the Intervention and Control Groups; and then for the Intervention Group
alone, looking at the effects of: the nature of the dwelling transition; different aspects
of dwelling change; changes in location; changes in neighbourhood quality; living in a
regeneration area; and the relationship between social outcomes and mental health
and well-being outcomes.
8.
A total of 50 qualitative interviews were conducted with Intervention Group
members after rehousing. In the first wave, 28 people were interviewed between one
and 3 years after they have moved into a new home. In the second wave, 22 people
were interviewed between 3.5 and 5 years after they had moved into a new home.
The interviews aimed to find out more detail about how moving home had impacted
upon respondents’ housing conditions, health and social and community outcomes.
9.
In design terms, SHARP’s value lies in the fact that it is a prospective,
controlled study across many different locations in Scotland. In practice, the study’s
high retention rate which maintains continuity of participants is also a strength.
SHARP looks at a range of outcomes, not simply health, and allows the investigation
of relationships between outcomes. The limitations of the study come from the fact
that it is modest in size and has a follow-up duration of only 2 years (even though this
is longer than in many other studies). The Intervention and Control Groups, whilst
matched on certain criteria, are not identical and differ in some aspects of residential
circumstances, which limits the ‘purity’ of the study.
2
CHAPTER ONE
BACKGROUND
Housing, health and regeneration
1.1
Public policy in the U.K. has recently demonstrated a strong interest in 3
areas:
•
•
•
Inequalities in health, particularly spatial or area-based inequalities;
Joined up policy which seeks to enhance the gains from particular
interventions by placing them in the context of other actions and supports; and
Measuring the impacts of policy and evaluating the evidence base for claims
about the gains from current and prospective interventions.
Inequalities in health
1.2
It is widely recognised that Scotland carries an especially high burden of ill
health for a developed nation, and there are persistent health inequalities between the
poorest and most well-off members of society. These inequalities can be seen not
only in terms of socio-economic groupings but also according to area of residence.
The Report of the Measuring Inequalities in Health Working Group (2004) also noted
for example urban/rural inequalities and between different ethnic groups.1
The
health gap between manual or routine occupations and professional or managerial
occupations in Scotland is reported to have widened over the last 2 decades.2
1.3
Concern over the existence inequalities has been mirrored by increasing
efforts to (i) understand the determinants of those inequalities, and (ii) find the means
of tackling such inequalities. In looking at the determinants of poor health and health
inequalities, researchers and policymakers alike have pointed to the importance of life
circumstances. Thus, it is now widely accepted that poor health is not solely a result
of individual behaviours and unhealthy choices (to smoke, to drink to excess, to eat
unhealthily) but is strongly determined by so-called “upstream” influences on health –
in particular the physical, social and legislative environment in which those choices
are made. Thus, a healthy diet is influenced by ones’ access to healthy, affordable
food; and good mental health is influenced not just by personal stresses (including
financial stresses) - though these are undoubtedly crucial – but also by healthpromoting and health damaging physical environments. In such an analysis, health
inequalities are seen as being the result of, or subject to, major upstream influences
(such as social and health policies) rather than solely the result of downstream
influences (unhealthy behaviours).
1.4
The corollary of this argument is that the means of tackling poor health and
reducing health inequalities also lies upstream. Thus, improving the health of the
public can be achieved through altering and improving living and working conditions;
in particular through improving the quality of housing and the quality of places - the
latter in terms of the upkeep/degraded nature of the local environment; the availability
1
2
http://www.scotland.gov.uk/Resource/Doc/47171/0013513.pdf
Layland et al., 2007 http://www.sphsu.mrc.ac.uk/files/File/reports/OP016.pdf
3
of services and facilities; the prevalence of health damaging or promoting behaviours;
and the degree of community cohesion and support. The physical fabric of housing is
also an influence on health, mediated by damp, warmth and cold3.
1.5
Poor housing can also impact on educational and other opportunities. This has
been recognised frequently in Government documents (e.g. “Poverty, poor housing,
homelessness and the lack of educational and economic opportunity are the root
cause of major inequalities in health in Scotland...”).4 By extension housing and
regeneration activities may provide a key opportunity for improving the health of the
public, a point made frequently over the past decade, and reiterated most recently by
the World Health Organisation5. The provision of good quality, affordable housing is
therefore widely seen as a “public health” intervention.
1.6
The policy response to the above has been, firstly, to extend the objectives of
housing and regeneration policies in order to seek health gains from these actions: the
UK Government has in the past said for example that "regeneration partnerships offer
significant opportunities for health gain". Secondly, there has been an emphasis on
area-based health interventions such as Health Action Zones and Healthy Living
Centres, and to some extent these have been targeted upon regeneration areas where
their impact might be greater.
1.7
The actual impacts of such interventions also remain a subject of interest.
Much previous research into the impacts of housing upon health have been concerned
with the impact of specific housing elements such as heating, insulation, and space.
Yet in the case of housing associations, at the time of the SHARP study, most housing
investment related to the provision of new, general needs housing, rather than
rehabilitation or improvement according to the Scottish Homes Investment
Programme 1999-2000. In other words, the bulk of the public housing investment in
this sector involved providing a new home to someone who previously was in housing
need.
1.8
There are other alterations to one’s circumstances which may accompany the
move to a new house. First, a change in housing provider and consequent changes in
housing management practices. These practices include landlords' powers (and
willingness) to deal with neighbourhood nuisance and anti-social behaviour. Second,
in getting a new house, people may also acquire a better local environment in terms of
its physical qualities, the provision of services and facilities and the level of
community activity and support. As well as being of policy interest in terms of
"healthy communities", community integration has been shown to be significantly
related to health.6 Any new research on housing and health needs to take these
changes into account.
3
Wilkinson, D. 1999 Poor Housing and Ill Health: a Summary of the Research Evidence. Edinburgh:
Central Research Unit, Scottish Executive.
4
Scottish Executive. Our National Health: A plan for action, a plan for change.
Edinburgh: Scottish Executive, 2000
5
In the recent report: Our Cities, Our Health, Our Future. Final Report of the Urban Settings
Knowledge Network. WHO Commission on the Social Determinants of Health. WHO 2008.
6
Lloyd-Williams et al. (In press) Eur J Public Health. Delivering a cardiovascular disease prevention
programme in the United Kingdom: translating theory into practice.
4
1.9
To summarise: most housing investment in the UK social rented sector results
in large-scale changes in housing conditions and possibly also in neighbourhood
circumstances. These changes have the potential to improve health and health
inequalities, and any evaluation of housing's impacts upon health needs to assess these
outcomes.
Previous research on housing and health
1.10 There is a “housing evidence base” which goes back many decades, and
includes many hundreds of surveys demonstrating the relationship between poor
quality housing and poor health7. It may seem surprising therefore that there is a need
for yet more evidence. However, much of the existing evidence is concerned with
demonstrating cross-sectional associations; there have been far fewer studies which
have demonstrated the health and other outcomes which follow from investing in
social housing – that is, evaluative rather than descriptive studies. The earliest
evaluation studies probably date from around the 1930’s, and were conducted in
Glasgow. In more recent years there have been several controlled trials, and, most
recently several, randomised controlled trials.8
1.11 A systematic review of intervention studies (carried out in 2001) found that
housing improvement may lead to small improvements in self-reported physical and
mental health and reductions in some symptoms, but noted that adverse effects on
health are also possible.9 However, the evidence is patchy and controlled study
designs are rare. Of the 18 studies identified in the review, 6 were prospective
controlled studies and only one was a randomised controlled trial. The most recent
study in this field (published in 2007), a large randomised controlled trial from New
Zealand, assessed whether insulating older houses increases indoor temperatures and
improves occupants' health and well-being. Its findings suggested that improving the
indoor environment may lead to improved self-rated health and reduced health service
use. Previous studies have also indicated that warmer and less humid living
conditions may improve health, but they also suggest that the health benefits
disappear if housing costs increase. The authors of the New Zealand study also
highlighted the almost complete lack of an evidence base for the effectiveness and
cost-effectiveness of public health and social interventions, and pointed to the need to
collect better evidence of the effects of interventions in the housing sector.
1.12 Housing professionals also recognise that there is a lack of evidence to back
up claims that housing investment produces health gains. One of The Chartered
Institute of Housing's Policy Officers commented in 1997 that "There is obviously still
much work to be done to produce convincing arguments for increasing expenditure on
housing in order to directly benefit the health of people and communities".10 Little
has changed in the subsequent years to alter this situation, and yet as policy initiatives
multiply to respond to health inequalities, the call for “better evidence” increases.
7
Wilkinson, D. 1999 Poor Housing and Ill Health: a Summary of the Research Evidence. Edinburgh:
Central Research Unit, Scottish Executive.
8
Thomson H et al. In press. Systematic review of housing and health: an update.
9
Howden-Chapman P et al. Effect of insulating existing houses on health inequality: cluster
randomised study in the community. BMJ 2007 doi: 10.1136/bmj.39070.573032.8
10
Housing and Health - Making the Links Count, Housing Review 46:3.
5
Regeneration and health
1.13 While research on components of housing improvement (such as insulation
installation) is important, new evaluative research on housing has to take into account
the fact that housing investment increasingly occurs within a wider context of
regeneration programmes. In the case of Scotland, we know that nearly 40% of the
housing output from Scottish Homes urban investment in the year 1999/2000 - when
initial plans for the SHARP study were made – were in the Priority Partnership Areas
(the main urban regeneration initiative prior to Social Inclusion Partnerships). But
this also meant that 60% of housing output was to occur either in urban areas not
subject to the main regeneration programmes, or in rural areas. This offered the
SHARP research team the opportunity to investigate whether housing investment in
the wider regeneration context produces health gains over and above housing
investment in itself; in effect a "natural" experiment.
1.14 By mounting a study of the social housing investment programme as a whole,
we were also able to overcome one of the limitations of past research on the effects of
wholesale housing change, namely that they have been focused on a single site or
development; this introduces contextual biases that limit the generalisation of the
findings. Rather, we wanted to conduct a multi-site study that would avoid this
limitation.
The need for an experimental study
1.15 When circumstances permit, any intervention should be evaluated in the form
of an experiment, in which individuals (or households) are randomly allocated to the
Intervention Group - which receive the new housing, or new intervention, or to a
Control Group - which may receive the intervention at a later point in time. When this
is not possible, for example when the researchers do not control the roll-out of the
intervention (e.g., a new housing building programme, or other new investment in
housing), then a controlled study is essential, to ensure that any change in health or
well-being can be identified over and above any changes in health which happen
naturally over time. This is sometimes referred to as an observational design, or a
quasi-experimental design.
1.16 The SHARP study used this design in that it followed, over a period of years,
the effects on health and well-being of people who moved into general-needs, newbuild accommodation provided by housing associations during 2002-2003. However,
as researchers it was not possible or practical for the SHARP team to control the
allocation of families to the new homes. Instead we identified a matched comparison
group whose health and other outcomes were followed up over time, in parallel with
the Intervention Group. The study design is described in more detail in Chapters 3 and
4.
6
Summary
•
There is ongoing policy interest in the social determinants of health and health
inequalities, and in the potential for housing and regeneration policies to improve
health and well-being.
•
Though there is a long history of research on housing and health, there are still
relatively few studies which explore the health impacts of large-scale changes in
housing conditions and in neighbourhood circumstances.
•
Previous studies suggest possible effects on physical and to a greater extent
mental health, although controlled studies are scarce.
7
CHAPTER TWO
RESEARCH AIMS AND OBJECTIVES
2.1
As noted in Chapter One, there have been several previous evaluative studies
of the health impacts of housing improvement (like SHARP), and these have
suggested that general housing improvements may result in small improvements in
physical health and general well-being; and mental health improves following housing
improvements, where the degree of mental health improvement may be linked to the
extent of the housing improvement. It has also been found that improved energy
efficiency measures may alleviate respiratory symptoms.
2.2
However much of the previous research has focused on specific health
problems, and housing conditions, at the expense of other aspects of the local
environment which may affect people’s health – such as crime, and other types of
anti-social behaviour. SHARP therefore set out to evaluate the effects on health and
well-being of new housing provision, and to explore some of the mechanisms by
which health is affected - in particular, how housing and other types of neighbourhood
change interact. We thus outline below the aims and objectives of SHARP with
respect to 2 main aspects of the intervention: firstly, change in housing circumstances,
and secondly, neighbourhood change.
Housing change and health
2.3
SHARP sought broad-ranging information concerning respondents’
knowledge, involvement, behaviour, opinions and perceptions of the neighbourhood
milieu, beyond the immediate home environment. In particular we aimed to examine:
•
The extent to which rehousing into a new socially rented dwelling delivers
improvements (or indeed decline) for occupants in terms of housing
conditions, neighbourhood conditions, housing management performance and
sense of community;
•
Whether these improvements are associated with changes in a person’s
neighbourhood and landlord as well as rehousing itself; and
•
To what extent people who are rehoused experience changes in their physical
health, health behaviours and mental health and well-being, and whether these
health changes are sustained over time.
2.4
To answer these questions we collected data from social tenants who were
rehoused into a new, general purpose socially-rented home developed and let by a
Registered Social Landlord (the “Intervention” group). These data were collected at 3
points in time: before moving, one year after moving, and 2 years after moving. In
total nearly 700 people were interviewed in the Intervention and Comparison groups.
Thus we have a quasi-experimental study (a prospective, controlled study) as
described in Chapter One. There is some debate over whether a matched comparison
group in an observational can be described as a “control” group. An alternative would
be to describe them as a “comparison” group. We will however retain the use of the
term “Control Group” in this series of reports, as it is widely understood and it
8
appears in some of the study materials (for example, the SHARP ethics application,
and in earlier reports to the funders).
Housing outcomes
2.5
To collect data on housing conditions, SHARP interviewers asked respondents
about 11 potential problems associated with their home (such as problems with damp,
keeping home warm in winter, etc.). These questions were asked again at follow-up at
one year after the move in the postal survey, and at final follow-up, approximately 2
years after the move. Control Group respondents were also asked these questions at
baseline, and at final follow-up. In particular we asked about the following problems:
•
•
•
•
•
•
•
•
•
•
•
Damp
Keeping home warm in winter
Not enough privacy
Problems getting in or out of home
Smells and fumes
Noise from other household members
Rooms too small
Accidents outside the home
Noise from neighbours
Accidents inside the home
Rooms too large.
2.6
By asking about these items at baseline and follow-up we could ascertain
whether housing conditions had actually changed over time in the Intervention Group
compared to the matched controls, and secondly whether this change was associated
with change in health and well-being. These issues were also discussed with selected
respondents in the in-depth qualitative interviews (see Chapter 7). Respondents were
also asked how frequently they had difficulty paying certain household charges.
Previous studies have suggested that some bills (e.g. fuel, and food bills) may become
more difficult to pay after moving home.
2.7
We also asked how people felt about their home (including any psychological
benefits they may derive), and about general changes in their lifestyle. We also asked
the Intervention Group about their experiences of moving, and their experiences of
living in temporary accommodation.
Health outcomes
2.8
We collected data on a range of self-reported health measures, including
health behaviours by means of the survey, which involved face to-face interviews at
baseline and 2 years in the Intervention and Control Groups, and by postal survey at
one year (in the Intervention Group only). In particular we asked about:
•
Self-reported health (‘In general, would you say you health is … excellent,
very good, good, fair, or poor?”);
9
•
•
•
•
•
•
•
•
•
•
Whether the respondent felt that any health problems (for them or any member
of the household) were due to their housing conditions;
Long standing illness (LSI);
Common symptoms (e.g., sleep problems, headache);
Respiratory conditions including asthma;
Smoking, diet, physical activity and alcohol consumption;
Child health (whether a child in the household had been diagnosed with
asthma, eczema, bronchitis, or other long-term illnesses or disabilities;
whether the child had suffered from a number of conditions within the last
calendar month, and the number of days off school the child had had within
the last month);
Health-related quality of life (using the SF-36 measure, which includes a
measure of emotional well-being);
Sense of control over circumstances;
Accidents in and outside the home; and
Use of health services (GP attendances, inpatient and outpatient admissions,
use of prescribed medications).
2.9
The majority of these data were collected by means of the survey. We also
asked respondents for permission to access routine NHS data on inpatient admissions,
which was given by the majority of respondents. We thus had some objective measure
of use of NHS services.
2.10 Our analyses of these data were driven by an analytical framework which was
we produced early in the project (available on request). This was intended to reduce
the likelihood of type 2 error by setting out a priori primary and secondary
hypotheses, as well as the causal pathways which we hypothesised to link the
different aspects of the intervention (that is, housing and neighbourhood change) and
the health and social outcomes.
Neighbourhood change and health
2.11 One’s social environment has been cited as a determinant of physical and
mental health; the quality and strength of one’s relationships for example has been
associated with subsequent mortality in several prospective studies.11 In investigating
neighbourhood change as a determinant of change in health, SHARP therefore needed
to assess change in the social aspects of the neighbourhood (such as changes in “sense
of community”), as well as physical changes in the neighbourhood which may or may
not be health-promoting.
2.12 The survey therefore assessed respondents’ satisfaction with the area in which
the household was then living, about specific potential problems with the area, and
about the friendliness of local people. Issues relating to the community were also
discussed in the in-depth qualitative interviews.
11
Stansfeld SA. Social support and social cohesion. In: Social Determinants of health by Marmot M &
Wilkinson R. OUP1999.
10
2.13 Overall we asked about anti-social behaviour (such as crime, vandalism and
graffiti); aspects of the physical environment and the provision of services (such as
street-lighting, public transport, play areas, and facilities for children/young people;
and traffic). We also asked respondents about their social functioning, their
participation in local activities, and relationships with others in the local community
(“bonding” social capital), and their sense of “belonging” to the community.
Health improvement in regeneration areas
2.14 SHARP also examined whether rehousing within a regeneration area (Social
Inclusion Partnership - SIPs)12 provides additional residential or health gains. Many
regeneration programmes – SIPs included – have aimed to promote social inclusion,
and in many cases promote health. However the extent to which they have achieved
these outcomes is unclear, partly because they have often not been measured.13 While
SHARP was not an evaluation of a SIP, just over half of the Intervention Group were
located in SIPs, offering the opportunity to explore whether location in a SIP or not
made any difference to the effects of the housing change. We also asked all
respondents whether they were aware whether their area was currently subject to a
major regeneration programme, and asked them whether there were any changes to a
number of listed services and facilities, which would be expected to change as a result
of SIP activities.
Socio-demographic and other data
2.15 Finally, in the survey we collected a range of data on individuals and their
households, including age, marital status, ethnicity, housing tenure and type,
household size and composition, employment and income, and aspects of household
expenditure. These allowed us to explore difference between subgroups in the study
as well as allowing us to assess the comparability of the Intervention and Control
Groups at baseline.
12
Social Inclusion Partnerships existed over the period 1996-2006 (some were established as Priority
Partnership Areas in 1996 and others as SIPs in 1999-2000) and were the main vehicle used by the
Scottish Government for channelling resources for the regeneration of communities. There were 34
area-based SIPs established across the country, and SHARP interviewed people in 22 of these. From
2005/6 onwards, SIPs were incorporated into the activities of Community Planning Partnerships.
13
Thomson H et al. (2006) Do urban regeneration programmes improve public health and reduce
health inequalities? A synthesis of the evidence from UK policy and practice (1980-2004). J Epidemiol
Community Health 2006;60(2):108-15.
11
Summary
Despite a long-standing policy and research interest in the question of the links
between housing and health, there is a need for new research for the following
reasons:
•
To assess the health impact of a comprehensive change in housing
circumstances, such as the provision of new housing;
•
To utilise objective measures of health rather than rely solely on self-reported
health;
•
To contrast the impact(s) of housing with and without a wider regeneration
context; and
•
To apply rigorous design and control procedures to the research methodology.
SHARP’s main aim therefore was to conduct a multi-method, multi-disciplinary study
investigating the impacts of new social housing on recipients, with the objectives of
investigating:
•
The extent to which rehousing into a new socially-rented dwelling delivers
improvements (or indeed decline) for occupants in terms of housing
conditions, neighbourhood conditions, housing management performance and
sense of community;
•
Whether these improvements are associated with changes in a person’s
neighbourhood and landlord as well as rehousing itself; and
•
The extent to which people who are rehoused experience changes in their
physical health, health behaviours and mental health and well-being, and
whether these health changes are sustained over time.
12
CHAPTER THREE
STUDY COMPONENTS
Survey methods
3.1
Many aspects of SHARP’s methods had been tested in a pilot study, funded by
Scottish Homes and conducted in 2000. This pilot study explored the possibility of
using waiting list controls, but we rejected this as infeasible, as tenants stay on
waiting lists for a year and longer. Other aspects of the study however appeared
feasible, and a questionnaire was also developed and piloted, which was subsequently
adapted for use in SHARP.
3.2
After ethical approval had been sought and granted by Glasgow University
Ethics Committee, and the Multi-Centre Ethics Committee for Scotland, SHARP
proceeded to recruit participants for up to 3 Waves of data collection. The methods
are described briefly below. The achieved samples at each Wave are described in
Chapter 5.
Wave 1: Pre-rehousing questionnaire survey
3.3
At Wave 1, Intervention Group members who were about to move house were
interviewed approximately 1-2 weeks before they moved into their new home: this
survey took place between May 2002 and December 2003; similar numbers of
Control Group members were interviewed between October 2002 and April 2004.
The interviews were conducted face-to-face by a specially trained group of
interviewers, and the sample included households with families (61% of the sample),
and older people (approximately 12%). Respondents from rural and urban areas were
selected for the survey, and households in Social Inclusion Partnership (SIP) and nonSIP areas were included. We also compared the data from those who were due to
move home with data from the Control Group of people residing in the same
destination locality, but who were not themselves rehoused from their existing
dwelling.
Wave 2: Post-rehousing postal survey
3.4
The aim of the postal survey at Wave 2 was to collect information from the
Intervention Group respondents one year after they had moved into their new
accommodation. This was undertaken using a self-completed postal questionnaire
accompanied by a pre-paid envelope, which was posted out to respondents.
Wave 3: Follow-up questionnaire survey
3.5
At Wave 3, approximately 2 years after the Intervention Group had moved
into their new accommodation, face-to-face interviews were carried out by the survey
team once again. These took place between May 2004 and April 2006. Interviewers
were provided with contact sheets one month before the due date of the interview and
were requested to make contact with the respondents who had previously received an
13
introductory letter. There was a period of 3 months (one month before the due date
and 2 months after) to achieve the interview, after which it was not deemed valid to
collect the data. An equivalent Control Group survey was also conducted at this time,
with interviews taking place between August 2004 and May 2006.
Qualitative in-depth interviews
3.6
We conducted 2 waves of qualitative interviews. The first wave was
conducted in 2005 (post-Wave 2) with a sample of intervention respondents one year
after they had been rehoused. Twenty-eight people were selected for interview; this
task was undertaken by 4 interviewers who were recruited on the basis of their
previous experience of qualitative interviewing. This meant that one interviewer who
had experience of detailed face-to-face quantitative interviewing research was not
considered suitable to continue with the qualitative study.
3.7
While there were some important insights from the qualitative interviews
conducted at one year, we felt that there was value in longer-term follow-up of
respondents. A second wave of qualitative interviews was therefore conducted postWave 2 in early 2008, to explore the longer-term impacts of moving house and
neighbourhood. Twenty-two interviews were achieved in this exercise.
Routine data sources
3.8
We also sought 2 other sources of data to link to the SHARP dataset. Firstly,
we sought respondents’ permission to obtain SMR01 data from the Information and
Statistics Division (ISD) of the then Scottish Executive. This dataset records
information about treatments and diagnoses relating to anyone who receives acute
inpatient or day-care in NHS hospitals or NHS contracted beds in other facilities.
Approximately 90% of each group in the study (234/262 in the Intervention Group;
256/285 in the Control Group) gave this consent. These data describe all non-obstetric
and non-psychiatric discharges, and lengths of stay. We obtained these for consenting
participants from 1997 to 2005/6, giving us data “before” and “after” the intervention.
3.9
Secondly, we obtained SCORE lettings codes for the Intervention Group. The
SCORE system provides data collected by the landlord on each tenancy (e.g., data on
number of children, income and benefits, and details of the accommodation).
14
Summary
•
SHARP involved 3 Waves of quantitative data collection: face-to-face
interviews at baseline in the Intervention and Control Groups, followed by a
postal survey which was administered to the Intervention Group only, one year
after they had moved home. A final face-to-face interview of both groups was
conducted at Wave 3 – approximately 2 years after the Intervention Group had
moved home. At all 3 Waves’ data were collected by means of a specially
developed questionnaire.
•
Two waves of in-depth qualitative interviews were also conducted, among
Intervention Group participants. These were conducted after Wave 2 (n = 28)
and after Wave 3 (n = 22).
15
CHAPTER FOUR
SAMPLING AND SURVEY
Scheme selection
4.1
When SHARP was initiated we envisaged that the Intervention Group would
consist of the new occupants of social housing provided through around 20-30
housing developments throughout Scotland, with the developments concerned
consisting of new, general-purpose housing built by registered housing associations.
It was intended that the geographical distribution of the intervention study sites would
be partly determined by the pattern of developments undertaken and completed over a
period of a year, although we would aim to select sites which offered a range of
locations in cities, towns and rural areas. In this way, the results would represent the
impacts arising from a diverse national programme of new social housing provision
over an annual cycle. A great number of housing associations with new developments
scheduled to come off site in the financial year 2002/3 were therefore contacted and
invited to take part in the study.
4.2
We broadly achieved our aim for a nationally representative sample of
housing developments, although the national programme involved fewer houses in
each development than we were originally led to believe. As a result, by the time the
Wave 1 survey had closed we were working with 45 Registered Social Landlords,
who were instituting around 57 housing developments.
Identification of Control Group and matching of households in the 2 study
groups
4.3
As noted earlier there is some debate over whether a matched comparison
group in an observational study can be described as a “control” group. A less rigid
description would be to describe them as a “comparison” group. However we have
retained the use of the term “Control Group” in this report, as it is widely understood,
is widely used in the literature and it appears in the study materials and in previous
reports. The Control Group were defined as tenants living in rented housing in the
same localities as the Intervention Group would be living in after rehousing, and who
were not about to be rehoused as far as we could tell14.
4.4
Great effort was made to ensure substantial and balanced samples in the
Intervention and Control Groups with respect to the key characteristics of region
(Unitary Authority), household type (family, adult or older person) and household
tenure (rented versus owned.)15. The Intervention Group respondents were selected
first and Control Group respondents were then selected to match the proportions of
Intervention Group respondents of each Unitary Authority, household type and tenure.
14
The vast majority of the Control Group were social sector tenants. The ‘same locality’ was defined
for us by the RSLs providing the newly built social housing being studied as part of the Intervention
Group sample.
15
We aimed to ensure that the Control Group were in rented accommodation, as the Intervention
Group would be after rehousing. In the event, 100% of the Control Group were tenants (97% social
renters, 3% private renters), as were 93% of the Intervention Group at Wave 1.
16
4.5
The household types were defined as follows: ‘Family households’, were
those containing children under the age of 16 years; ‘Older person households’, were
those in which the respondent and adult members of the household were of
pensionable age; ‘Adult households’ were those in which a combination of
relationships was possible, including parents with children over 16 years of age,
people unrelated to one another, couples etc. Subsequently, in the analysis phase, we
re-classified some of the adult households as ‘families with non-dependent children’,
based on intra-household relationships.
4.6
There were other characteristics of the population for which it would be
beneficial for there to be a balance between Intervention and Control samples: further
geographical descriptors (urban/rural location, presence or not in a SIP),
accommodation type (house, flat, etc.), number of people in the household and
demographic aspects of the respondents (age, sex, marital status, ethnicity). However,
a requirement for a balance of these characteristics was not imposed at the time of
selection so that the process did not become so strict that insufficient “suitable”
respondents could be identified. The balance of these aspects arising in the course of
sample balancing with respect to region, household type and tenure is examined in
Chapter 5.
Participant recruitment
4.7
Interviews with Intervention Group participants took place between the date
on which housing associations knew (i) the date on which the key to the new property
would be handed over to the new residents and (ii) the date on which the list of new
residents was finalised. The usual pattern was for Registered Social Landlords (RSLs)
to have both items of information some 3 weeks before the date of removal. Tenants
would be contacted by the RSLs during this period with information about SHARP,
about participation, and were allowed to indicate their willingness to participate. They
were then contacted by a named interviewer from MRC SPHSU, and a suitable date
for an interview was arranged. The interview could therefore take place up to 3
months before the removal of the respondent to their new home.
4.8
This recruitment procedure posed several difficulties for the conduct of the
survey, the main ones being: (i) notification of names by the RSL at very short notice
before the tenant was to move; (ii) the use of opt-in versus opt-out consent for
participating tenants; (iii) delays to new-builds; and (iv) a lack of enthusiasm for the
study as a whole on the part of some RSLs, despite significant efforts being made by
the researcher to work with the RSLs to minimise the burden and to “sell” the study
(including workshops to present early findings). Other RSLs were very positive about
the study and keen to learn more about the health and well-being impacts of their
housing.
Participant retention
4.9
As noted elsewhere, once respondents had agreed to take part, retention to the
study was excellent, with high participation rates at Wave 2 (78%) and Wave 3 (78%
17
Intervention Group; 73% Control Group). This was mainly due to the hard work of
the survey team and interviewers who established a rapport with respondents and
maintained contact with them between Waves – for example, by means of yearly
Christmas cards and calendars, and full-colour newsletters to SHARP participants
(survey respondents, fieldworkers, and RSLs) reporting on the findings.
Survey management and conduct
4.10 The fieldwork was managed by the MRC SPHSU survey team. After an initial
unsuccessful trial with a market research company, we found that the most effective
means of achieving interviews was to provide our own specially trained group of
interviewers. The survey team recruited and trained the interviewers, and took
responsibility for health and safety issues (including conducting risk assessments).
They also established that the questionnaire could, if necessary, be completed over the
telephone so that interviews with more remote communities (e.g., Shetland) could be
included with minimum effort and expense. However, most of the more distant
interviews were conducted by a firm local to the Grampian region, called ‘Business
Plus Scotland’; the firm’s input was again contracted and managed by the MRC
SPHSU Survey Team.
4.11 Changes to procedures were made as the study proceeded. For example,
single-interviewer recruitment of controls raised safety concerns and also because of
intra-community contact, by which people within a community were forewarned of
the interviewer’s presence and sometimes encouraged not to participate in the survey.
Thus, interviewers were organised to go out to control locations in groups, rather than
individually. This approach proved more successful.
4.12 For the postal survey the questionnaire was sent to all members of the
Intervention Group one year after they had moved house. The high response rate was
achieved by means of close monitoring of replies and by using interviewers to collect
unreturned questionnaires in person. Experience with this and with previous surveys
has shown that some sort of personal contact is effective in achieving a high
completion rate. Individual collection accounted for 19% of the returned
questionnaires.
4.13 Once the final questionnaires had been received for each wave they were
checked briefly in-house and then sent out for data entry, by either Lindata Services
Ltd, or the Robertson Centre for Biostatistics at the University of Glasgow. When the
datasets were returned, extensive cleaning and logic checks then took place in-house
before analysis commenced. Quality control checks comparing the responses recorded
on the questionnaire with the corresponding values in the dataset were also carried out
on a minimum of 5% of questionnaires for each of the Intervention and Control
samples for all 3 Waves.
18
Summary
•
Intervention and Control Groups were matched on household tenure, area and
household type.
•
The fieldwork was managed by MRC SPHSU. It included training of a
dedicated SHARP interview team, who received regular feedback on the
conduct of the study and the findings.
•
Efforts were made to retain participants, and in fact retention rates were very
good, with 78% retained at Wave 2 and 78% (Intervention) and 73% (Control)
by Wave 3.
•
Extensive checking of data quality was conducted before analysis took place,
including checking a 5% sample of the original questionnaires.
19
CHAPTER FIVE
ACHIEVED SAMPLES
5.1
This section describes the main characteristics of the achieved sample at
Waves 1 and 3, first overall, then in terms of the geographical distribution of the
sample (Unitary Authority, urban/rural classification, SIP status), the characteristics
of their housing (household tenure, and accommodation type) and household
(household type, number of people in the household) and the demographics of
respondents (age, sex, marital status, ethnicity), comparing the Intervention and
Control Groups in each case.
5.2
Considerable effort was made at the time of recruitment to Wave 1 to ensure
that the proportions of the different classes of each characteristic were roughly
comparable in the Control and Intervention Groups (obviously, it was not possible to
control these for Wave 3). We examine the degree of success of this here statistically
by log-linear modelling of the frequencies of households in each combination of
Group, Wave and characteristic in question. Under these circumstances, nonsignificance indicates that the desired similar distributions of respondents across the
combinations of classes had been achieved.
Overall samples
5.3
At Wave 1, interviews suitable for analysis were successfully conducted with
334 respondents in the Intervention Group and 389 in the Control Group. The 334
Intervention Group interviews represent a 46% response rate16, and approximately
19.5% of all occupants of the 57 schemes included in the study. The Intervention
Wave 1 sample also represents around 10% of the annual national output of generalpurpose social rented housing within the RSL sector in Scotland17. The 389 Control
Group interviews at Wave 1 represents a 39% response rate18. At Wave 2, postal
questionnaires were returned by 280 Intervention Group respondents, representing a
response rate of 84%. Of those respondents interviewed at Wave 1, 262 (78.4%) of
the Intervention Group and 285 (73.0%) of the Control Group were interviewed again
at Wave 3.
Geographical distribution of the SHARP sample
Unitary Authority
5.4
Geographical region by Unitary Authority was one of the characteristics for
which a match between the Intervention and Control samples was attempted. At
16
723 of those people who were rehoused in the schemes included in the study were approached in
order to obtain the 334 interviews.
17
Data on the national social housing development programme reported an output of 3,200 units in
2000/1 and of 3,700 units in 2001/2.
18
A total of 1920 addresses/householders were approached for inclusion in the Control Group, of
which 920 (48%) were deemed ineligible by virtue of housing tenure or household type (given that we
were attempting to match the Intervention Group according to a 3-fold household-type classification).
Of the remaining 1,000 householders, 390 (39%) agree to be interviewed and 610 (61%) refused.
20
Wave 1 Intervention respondents were drawn from 21 Scottish and one English
Unitary Authorities; Control respondents came from 20 Scottish Unitary Authorities
(Table 5.1). At Wave 3, both the Intervention and Control Groups included
representatives from 20 Scottish Unitary Authorities. The largest single group of
respondents was based within Glasgow City Council boundaries for the 2 Groups at
both Waves (32-36%). This group was approximately 3 times the size of the next
largest cluster of respondents, who were in Perth and Kinross. Where clusters were
not specifically linked to an urban centre, e.g. Dundee City, they tended to represent
single urban areas within Unitary Authorities, such as Livingstone in West Lothian
and Paisley in Renfrewshire, since they are closely linked with individual RSLs.
5.5
Control respondents were sourced from the areas to which Intervention
respondents were moving, rather than matching Controls against the original
addresses of the Intervention Group. Nevertheless, the proportions of respondents
from each Unitary Authority were similar in the 2 Groups (χ2 = 18.175, d.f. = 22, p =
0.696) and between Waves (χ2 = 18.175, d.f. = 22, p = 0.696); the interaction between
Group and Wave also indicated no dissimilarities in proportions (χ2 = 2.407, d.f. = 22,
p < 1.000).
Table 5.1
Distribution of respondents by Unitary Authority, for Intervention
and Control Groups at Waves 1 and 3
Wave 1
Wave 3
Unitary Authority
Intervention
Control
Intervention
Control
Freq
%
Freq
%
Freq
%
Freq
%
Aberdeen City
13
3.89
20
5.14
13
4.96
14
4.96
Aberdeenshire
8
2.40
4
1.03
4
1.53
3
1.06
Argyll and Bute
6
1.80
8
2.06
2
0.76
4
1.42
Clackmannanshire
1
0.30
0
0.00
0
0.00
0
0.00
Dundee
20
5.99
23
5.91
19
7.25
20
7.09
E Ayrshire
1
0.30
0
0.00
1
0.38
0
0.00
Edinburgh City
4
1.20
4
1.03
3
1.15
4
1.42
Fife
9
2.69
5
1.29
7
2.67
4
1.42
Glasgow City
120 35.93
136 34.96
87 33.21
90 31.91
Highland
6
1.80
6
1.54
6
2.29
6
2.13
Inverclyde
4
1.20
3
0.77
3
1.15
1
0.35
N Ayrshire
1
0.30
3
0.77
1
0.38
3
1.06
N Lanarkshire
7
2.10
10
2.57
4
1.53
8
2.84
Perth and Kinross
33
9.88
37
9.51
26
9.92
31 10.99
Renfrewshire
23
6.89
27
6.94
20
7.63
18
6.38
S Ayrshire
5
1.50
7
1.80
4
1.53
7
2.48
S Lanarkshire
15
4.49
23
5.91
12
4.58
15
5.32
Shetland
2
0.60
2
0.51
2
0.76
1
0.35
Stirling
17
5.09
21
5.40
15
5.73
16
5.67
W Dunbartonshire
15
4.49
16
4.11
12
4.58
13
4.61
W Lothian
23
6.89
34
8.74
21
8.02
24
8.51
Manchester
1
0.30
0
0.00
TOTAL
334
389
262
282
Note to table: Values are for cross-sectional samples at each Wave
21
Urban/rural classification
5.6
The urban/rural status of the population in the SHARP survey, for which a
match between the Intervention and Control samples was desirable but not directly
sought, was determined using the then Scottish Executive Urban Rural Classification
(Table 5.2). At Waves 1 and 3 the Intervention and Control sub-populations both
predominantly consisted of households in urban areas (almost 80%), and more than
half of the households were situated in large urban areas. Consequently, frequencies
for the remaining 4, non-urban classes were lower, and negligible in the case of small
remote towns. This significant difference in the distribution of the households (χ2 =
1382.8, 5 d.f., p < 0.001) largely reflects the intrinsic differences in population density
by which the classes are defined. Even so, the SHARP population over-represents the
large urban population and under-represents the number living in other urban areas
and accessible small towns because of the demands of the survey. It was not intended
to be a survey of the general population, but rather of those about to move into
general new-build accommodation provided by Housing Associations, and thus
reflects the differential distribution of Housing Associations across Scotland. It is
therefore more useful for us to examine the population of Scotland broken down by
housing association. The proportion of the rental market represented by Housing
Associations and Co-operatives in urban areas (20%) was twice as large as that in
rural areas (10%).19
5.7
In the event the proportions of the urban/rural categories in the Intervention
and Control Groups at Waves 1 and 3 of the survey were well matched. There was no
significant difference either in the proportions of households in each category
between the Intervention and Control Groups (χ2 = 4.162, 5 d.f., p = 0.451) or
between Waves (χ2 = 3.908, 5 d.f., p = 0.563); the interaction between Group and
Wave for urban/rural category was also non-significant (χ2 = 6.788, 5 d.f., p = 0.327).
Table 5.2
Distribution of households in Intervention and Control samples by
Scottish Executive Urban/Rural Classification (percentages)
Wave 1
Wave 3
Intervention
Control
Intervention
Control
Urban/rural class
(n = 333)
(n = 389)
(n = 262)
(n = 282)
Large urban area
58.9
55.0
54.2
53.5
Other urban
19.8
23.1
21.4
23.0
Small accessible town
5.4
2.8
3.4
3.5
Small remote town
0.6
0.8
0.0
0.7
Accessible rural
7.8
11.1
13.7
11.3
Remote rural
7.5
7.2
7.3
7.8
Note to table: Percentages are for cross-sectional samples at each Wave
Location within SIP area
5.8
Whether respondents lived in accommodation located within a SIP or not was
the second geographical descriptor for which a match between the Intervention and
Control samples was desirable but not directly sought. In the event, between 44% and
19
SHS: www.scotland.gov.uk/stats/bulletins/00257-04.asp
22
51% of respondents in the Intervention and Control Groups at Wave 1 and Wave 3
lived in a dwelling located within in a SIP area. Twenty-two SIPs were featured to a
greater or lesser extent, 9 of which were in Glasgow City: Castlemilk, Drumchapel,
Glasgow East End, Glasgow Smaller Areas, Gorbals, Greater Easterhouse, Greater
Pollok, North Glasgow and Springburn. Together the Glasgow SIPs accounted for
30-34% of the households in the 4 Intervention-Control by Wave 1 - Wave 3 samples
and around 66.9% of those households located in a SIP. This is partly a reflection of
the geographical spread of SIPs across Unitary Authorities in Scotland, but is also a
product of the relationship between RSLs and regeneration projects. Other than
Glasgow, the Unitary Authorities of Dundee, Renfrewshire and West Dunbartonshire
contained about 25% of the remaining SIP-based households at both Waves. The
population not living in a SIP at the time of both Waves of the survey were most
predominantly located in the Unitary Authorities of Aberdeen City, West Lothian,
Stirling, and Perth and Kinross (53.1% and 53.9% of non-SIP-based respondents at
Waves 1 and 3, respectively). An additional 9.2% of the non-SIP-based population at
Wave 3 lived in Renfrewshire.
Table 5.3
Distribution of respondents by Social Inclusion Partnership, for
Intervention and Control Groups at Waves 1 and 3
SIP status and name
Wave 1
Wave 3
Intervention
Control
Intervention
Control
Freq
%
Freq
%
Freq
%
Freq
%
164 49.10
205 52.84 138
52.67 158
55.83
170 50.90
183 47.16 124
47.33 125
44.17
Not in SIP
In SIP
Individual SIPs
Alloa
1
0.30
0
0.00
Blantyre North Hamilton
4
1.20
11
2.84
Cambuslang
1
0.30
0
0.00
Castlemilk
8
2.40
13
3.35
Drumchapel
15
4.49
17
4.38
Dundee 1
18
5.39
14
3.61
Edinburgh Strategic
2
0.60
0
0.00
Girvan Connections
1
0.30
1
0.26
Glasgow East End
6
1.80
8
2.06
Glasgow Smaller Areas
2
0.60
0
0.00
Gorbals
4
1.20
13
3.35
Great Northern Partnership
0
0.00
2
0.52
Greater Easterhouse
29
8.68
28
7.22
Greater Pollok
9
2.69
10
2.58
North Ayr
1
0.30
0
0.00
North Edinburgh
0
0.00
1
0.26
North Glasgow
41 12.28
28
7.22
North Lanarkshire
1
0.30
1
0.26
Paisley Partnership
12
3.59
17
4.38
Springburn
0
0.00
5
1.29
West Dunbartonshire
14
4.19
14
3.61
Westerhailes
1
0.30
0
0.00
Note to table: Values are for cross-sectional samples at each Wave
23
0
7
0
6
14
19
1
0
5
0
5
0
21
6
0
1
27
0
0
0
12
0
0.00
2.67
0.00
2.29
5.34
7.25
0.38
0.00
1.91
0.00
1.91
0.00
8.02
2.29
0.00
0.38
10.31
0.00
0.00
0.00
4.58
0.00
0
6
0
10
11
10
0
1
5
0
10
1
21
9
0
1
16
1
11
2
10
0.00
0.00
2.12
0.00
3.53
3.89
3.53
0.00
0.35
1.77
0.00
3.53
0.35
7.42
3.18
0.00
0.35
5.65
0.35
3.89
0.71
3.53
0.00
Housing characteristics of sample
Housing tenure
5.9
Unlike other characteristics for which a balance was attempted between
proportions of respondents in the Intervention and Control Groups, housing tenure
was of most relevance after the move. Thus, although the great majority of
respondents in the 2 groups, by the very nature of the SHARP study, would be in the
social rented sector at Wave 3 (and likewise for the Control Group at Wave 1, since
these respondents did not move), the tenure of the Intervention Group at Wave 1 was
not of particular concern. Table 5.4 illustrates the distribution of tenures in the 2
Groups at both Waves.
5.10 Before rehousing, a little over a quarter of the Intervention Group were living
in the private sector, whereas 2 years after rehousing, 98.1% were in the social sector,
with a few people having moved into the private sector, for example through right-tobuy.
5.11 The Control Group, as per the study design, were overwhelmingly social
sector residents at Wave 1 (98.1%), and predominantly remained so at Wave 3
(88.1%). The movement of 9% of social renters into the private sector between waves
was, again, probably due to the right-to-buy. As a consequence, there was a
significant different in the proportions of owner-occupied, private-rented and socialrented tenures between the Intervention and Control Groups at Wave 3 (χ2 = 21.411,
d.f. = 2, p < 0.001). Consequently, there is some mismatching by tenure at Wave 3,
but this could not have been foreseen and controlled.
Table 5.4
Distribution of respondents by housing tenure, for Intervention
and Control Groups at Waves 1 and 3 (percentages)
Intervention Group
Control Group
Wave 1
Wave 3
Change
Wave 1
Wave 3
Change
Owned with mortgage
6.9
0.8
-6.1
0.0
6.0
+6.0
Owned outright
2.1
0.4
-1.7
0.0
2.1
+2.1
Private rented
10.5
0.4
-10.1
1.8
1.4
-0.4
Rented from family/friend
1.2
0.0
-1.2
0.0
0.7
+0.7
Other
7.8
0.4
-7.4
1.0
1.8
+0.8
TOTAL: private sector
28.5
2.0
-26.5
2.8
12.0
+9.2
Council rented
28.7
0.0
-28.7
49.3
35.9
-13.4
HA/Co-op rented
42.5
97.7
+55.2
47.8
51.8
+4.0
Com Scot /Scot Homes
0.3
0.4
+0.1
0.0
0.4
+0.4
TOTAL: social sector
71.5
98.1
+26.6
97.1
88.1
-9.0
Note to table: Base: Cross-sectional samples; Figures are column percentages (i.e., within wave).
24
Dwelling types
5.12 Dwelling types were too diverse to merit specific matching between
Intervention and Control Groups (Table 5.5). Moreover, since the very nature of the
SHARP project involved the study of the move of Intervention households
predominantly into new-build houses, similar proportions of dwelling types at Wave 1
were not required. The Control Group was split roughly evenly between flats and
houses at Wave 3, but as a result of rehousing, the Intervention Group changed from
mostly living in flats, to mostly living in houses: at Wave 3, these respondents were
divided 60:40 between houses and flats, which are significantly different proportions
from the Control Group (χ2 = 7.001, d.f. = 1, p = 0.008).
Table 5.5
Distribution of respondents by dwelling type, for Intervention and
Control Groups at Waves 1 and 3 (percentages)
Dwelling type
Detached house
Semi-detached
house
Terraced house
TOTAL: houses
Wave 1
Intervention
Control
(n = 333)
(n = 389)
4.8
0.3
12.3
21.6
11.1
28.2
Wave 3
Intervention
Control
(n = 262)
(n = 284)
1.9
0.7
35.1
18.7
26.7
48.6
Cottage flat
10.2
15.7
High-rise flat
5.7
0.3
Low-rise flat
21.3
21.9
Traditional
30.6
9.3
tenement
Flat in a
0.9
0.3
conversion
3.0
4.1
Other
TOTAL: flats
71.7
51.6
Note to table: Base: Cross-sectional samples; Figures are
wave).
26.0
63.0
32.4
51.8
12.6
0.8
18.3
0.4
16.2
0.0
24.3
3.9
0.0
1.1
5.0
2.8
37.1
48.3
column percentages (i.e., within
Household characteristics of sample
Household type
5.13 About 72% of the Intervention Group households were family households at
Wave 1, but by Wave 3 this figure had reduced to 57%, in comparison with the
Control Group, which experienced a smaller change from 63% to 57% of family
households (Table 5.6). Conversely, the percentage of older person households at
Wave 3 was approximately twice that at Wave 1 for both groups. Nevertheless,
neither of these differences was statistically significant (Intervention: χ2 = 5.119, 2
d.f., p = 0.077; Control: χ2 = 0.012, 2 d.f., p = 0.944). However, the Intervention and
Control Groups differed significantly in the proportions of household types at both
Waves (Wave 1: χ2 = 21.109, 2 d.f., p < 0.001; Wave 3: χ2 = 7.728, 2 d.f., p = 0.021),
so they cannot be considered to be fully balanced with respect to this characteristic.
25
Table 5.6
Distribution of respondents by household type, for Intervention
and Control Groups at Waves 1 and 3 (percentages)
Household type
Family
Adult
Older
Wave 1
Intervention
Control
(n = 334)
(n =389)
71.6
63.0
18.3
24.8
10.2
12.2
Wave 3
Intervention
Control
(n =262)
(n = 284)
56.8
56.7
21.9
22.2
21.3
21.1
Number of people in household
5.14 The distributions of numbers of people living in Intervention and Control
households were considered separately for the 3 household types (family, adult, older
person) since these 2 characteristics are related to some extent (Table 5.7). Family
households would be expected to be the most prone to showing differences in size.
However, there was little difference between the mean family size at Waves 1 and 3
either in the Intervention Group (3.51 vs. 3.43 people, respectively), or the Control
Group (3.41 vs. 3.50 people, respectively). The adult and older person households
consisted of slightly fewer than 2 people per household on average in both the
Intervention and Control Groups and at both Waves.
5.15 The distributions of household sizes were similar in the Intervention and
Control Groups for each household type at both Wave 1 (family: χ2 = 4.506, 4 d.f., p
= 0.342 [top 3 size categories combined]; adult: χ2 = 7.488, 3 d.f., p = 0.058; older
people: χ2 > 0.000, 1 d.f., p = 0.983) and Wave 3 (family: χ2 = 3.278, 5 d.f., p = 0.657
[top 3 size categories combined]; adult: χ2 = 2.493, 2 d.f., p = 0.287; older people: χ2
= 0.151, 1 d.f., p = 0.698). This suggests that the Intervention and Control samples
were well matched at both Waves with respect to this characteristic.
26
Table 5.7
Distribution of household size by household type, for Intervention
and Control Groups at Waves 1 and 3 (percentages)
Wave 1
Wave 3
Number
Household type of people Intervention
Control
Intervention
Control
Family
1
0.00
0.00
0.00
0.62
2
19.25
23.08
23.03
19.88
3
33.47
37.10
33.94
37.27
4
29.71
21.27
26.06
21.12
5
12.13
13.12
10.91
13.04
6
3.77
2.71
5.45
3.73
7
0.84
1.36
0.61
3.11
8
0.84
1.36
0.00
1.24
TOTAL
239
221
165
161
Adult
1
68.85
57.65
69.23
61.90
2
26.23
42.35
30.77
34.92
3
3.28
0.00
0.00
3.17
4
1.64
0.00
0.00
0.00
TOTAL
61
85
65
63
Older person
1
58.82
59.04
62.50
58.33
2
41.18
40.96
37.50
41.67
TOTAL
34
83
32
60
Respondent demographics
Age
5.16 The youngest person interviewed in the study was aged 18 years old and the
oldest was 94 years of age (at Wave 1). The median ages of the two populations (41
and 47.5 years for the Intervention and Control samples, respectively) were
significantly different (Mann-Whitney U = 46,225.5, p < 0.001), reflecting the
younger age distribution of the Intervention Group (Kolmogorov Smirnov Z = 2.253,
p < 0.001). This difference perhaps results from a general lessening of enthusiasm to
move with age. Since the inclusion of a household in the Control Group was
contingent on it not moving, there may have been an intrinsic bias in favour of
selection of older respondents. The attrition within the samples between Wave 1 and
Wave 3 had no substantial influence on the age distribution of the respondents.
Sex
5.17 Table 5.8 summarises the percentages of men and women in the 3 household
groups for the Intervention and Control samples (based on the full sample at Wave 1).
Overall, women were more likely than men to be respondents (73.0% vs. 27.0%),
particularly if the household was a family (82% of respondents in the Intervention and
Control samples).
5.18 In addition, women were significantly more strongly represented in the
Intervention than Control sample (76.9% vs. 69.6%; χ2 = 4.885, 1 d.f., p = 0.027).
27
Closer inspection reveals this may largely be ascribed to the older person households,
in which 80% of Intervention respondents were women compared with only 55.4% in
the Control sample (older person: χ2 = 5.916, 1 d.f., p = 0.015), rather than to the
other household types (family: χ2 = 0.001, 1 d.f., p = 0.977; adult: χ2 = 0.293, 1 d.f., p
= 0.588). These patterns were not substantially altered at Wave 3 as a result of the
attrition among the sample.
Table 5.8
Sex of respondent by household type, for Intervention and Control
Groups (Wave 1)
Household type Sex
Family
Male
Female
TOTAL
Adult
Male
Female
TOTAL
Older person
Male
Female
TOTAL
All
Male
Female
TOTAL
Intervention
Freq
%
43
18.0
196
82.0
239
27
44.3
34
55.7
61
7
20.6
27
79.4
34
77
23.1
257
76.9
334
Control
Freq
%
39
17.9
179
82.1
218
41
48.8
43
51.2
84
37
44.6
46
55.4
83
117
30.4
268
69.6
385
Marital status
5.19 The marital status of the respondents at the beginning of the study was
specifically sought at Wave 3. The information is presented here (Table 5.9) by
household type. The Intervention and Control samples both featured notably higher
percentages of ‘single’ respondents, and thus lower percentages of married and
cohabiting respondents, than the equivalent values for Scotland as a whole.
5.20 In both the Intervention and Control Groups, only a minority of the family
households (31% of the Intervention Group and 37% of the Control Group) were
headed by a married (or remarried) couple. As might be expected, adult households
featured the highest proportion of divorced respondents (over a quarter of Intervention
and Control cases). Likewise, older person households unsurprisingly contained the
highest proportion of widowed respondents (36% and 38% for the Intervention and
Control Groups, respectively).
5.21 The proportions of respondents of all classes of marital status classes were
similar in the Intervention and Control Groups for the family (χ2 = 10.742, 6 d.f., p =
0.097) and older person (χ2 = 6.178, 5 d.f., p = 0.289) households. However, adult
households in the Intervention Group had significantly more separated people and
fewer single and married people than those of the Control Group (χ2 = 12.599, 5 d.f.,
p = 0.027).
28
Table 5.9
Household type
Family
Adult
Older person
Marital status of respondent by household type, for Intervention
and Control Groups at Wave 1 (retrospectively)
Marital status
Single (never married)
Married (1st marriage)
Remarried
Separated (still legally married)
Divorced
Widowed
Other
TOTAL
Single (never married)
Married (1st marriage)
Remarried
Separated (still legally married)
Divorced
Widowed
TOTAL
Single (never married)
Married (1st marriage)
Remarried
Separated (still legally married)
Divorced
Widowed
TOTAL
Intervention
Freq
%
55
34.2
46
28.6
3
1.9
21
13.0
25
15.5
2
1.2
9
5.6
161
17
27.4
11
17.7
1
1.6
12
19.4
17
27.4
4
6.5
62
2
6.5
13
41.9
1
3.2
4
12.9
0
0.0
11
35.5
31
Control
Freq
%
43
27.0
52
32.7
7
4.4
17
10.7
29
18.2
8
5.0
3
1.9
159
23
37.1
16
25.8
4
6.5
2
3.2
16
25.8
1
1.6
62
6
10.3
26
44.8
1
1.7
1
1.7
2
3.4
22
37.9
58
Ethnicity
5.22 Seven different ethnic groups were identified across the Intervention and
Control populations. However, the actual numbers involved are very small with 2.1%
of the Intervention Group and 1.8% of the Control survey group from non-white
ethnic populations, which broadly approximates the ethnic make-up of Scotland as a
whole.
29
Summary
•
SHARP is a study of those people being rehoused into newly built social
rented housing, not a study of the general population. In relation to the latter,
the SHARP population over represents the large urban population and under
represents people living in other urban areas and accessible small towns. This
is largely a reflection of the differential distribution of Housing Association
activity across Scotland.
•
The SHARP Intervention Group sample of 334 tenants represents around a
10% sample of the annual national output of general purpose social rented
housing in Scotland in the RSL sector at the time. The largest single group of
respondents (a third of the sample) was based within Glasgow City Council
boundaries.
•
Some 44% and 51% of the Intervention and Control Groups were located in
SIPs at Wave 1 or Wave 3.
•
The distributions of Intervention and Control cases were generally well
matched, given the difficulties of recruitment. However there were differences
between groups in terms of household type, age and accommodation type.
•
Overall retention was good across the 3 Waves of the study.
30
CHAPTER SIX DATA MANAGEMENT AND STATISTICAL
ANALYSES
Data management
6.1
The file containing all data from the 3 Waves of the study was created from
text files supplied by the data entry companies, then developed, cleaned and
maintained using SPSS versions 12-15.
6.2
Data were supplied by the data entry companies as separate files for the
Intervention and Control samples. These were added to, or merged with, the main
datafile as soon as possible after they had been received to produce a wide-format file
with one case (respondent) per row and each questionnaire variable from each wave
in a separate column.
6.3
Data cleaning involved the following: a quality control check against the
original paper questionnaires of 5% of the sample; variable range checks (to identify
impermissible values of individual variables); and logical checks (to identify
incompatible combinations of values of different variables). The quality of data entry
was generally very high, and although cleaning was most extensive for the Wave 1
data, this resulted in changes to only 1.5% and 2.5% of the entries for the Intervention
and Control Group datafiles respectively. Many of these changes were made to
clarify missing, “don’t know”, “refused” and “not applicable” codes within a dataset
and to make codes consistent between the Intervention and Control samples. Most of
the definite errors concerned a small number of questions for which the coding frame
was particularly complicated and therefore prone to mis-keying during data entry.
Data cleaning for Waves 2 and 3 was completed considerably more rapidly as a result
of changes to the layout of the questionnaire in the light of the shortcomings
encountered at Wave 1, the existence of a single version for each of the Intervention
and Control questionnaires, the greater experience and thorough training of the
interviewers, which ensured fewer mistakes and greater consistency in recording
responses at the time of interview, and the thorough checking of questionnaires before
data entry. Apart from one “frameshift” error in the version of the Wave 3 Control
data that was supplied to us, errors were negligible at Waves 2 and 3.
6.4
The main tasks of developing the file of the 3 Wave’s data were to ensure
correct and convenient labelling of variables (to simplify subsequent programming of
analytical syntax) and the consistency of the category values within them, and to
incorporate additional relevant data not collected in the questionnaire (e.g., unitary
authority, urban/rural classes, SIP status, etc.), and to derive additional variables.
Derivation of novel variables
6.5
New variables were developed from the existing data to enable the originally
intended analyses and to pursue other lines of investigation suggested by them.
Several key independent variables were permanently incorporated into the main
datafile since these were commonly used in many analyses (e.g., location change
groups, classes of landlord and tenure change between Wave 1 and Wave 3,
31
household type reclassified as families with or without dependent children, adult, or
older person households).
6.6
Novel outcome (dependent) variables were derived for the specific analyses
and were not habitually retained in the main datafile once they had been analysed.
These new variables were of several general types: recoded categorical variables,
aggregate scales, and transformed continuous variables.
6.7
Categorical (nominal and ordinal) variables were sometimes recoded as
considered appropriate to combine classes, either to ensure adequate minimum sample
sizes within cells or to simplify analyses (e.g., 5- or 7-point Likert scales were often
reduced to 3-point scales or even dichotomous variables).
6.8
To produce a straightforward combined analysis of groups of related variables
(e.g., the 28 specified aspects of the home that could have been identified as a
problem) aggregate scores were calculated that took into account the number of items
cited, weighted by the strength of opinion relating to them. (Sometimes it was
appropriate to include “don’t know” responses in these calculations.) Typically these
were transformed to a standardized scale from 0 to 100 for ease of interpretation and
also to adjust for aggregate scores based on different numbers of items (for example, a
household with no gas supply could not cite this not-applicable item as a problem, so
the scale was calculated on the basis of a denominator of 27, rather than 28).
6.9
Continuous outcome variables that deviated strongly from a normal
distribution were normalised by appropriate transformations to enable parametric
analyses to be carried out.
Statistical analyses
6.10 The analyses presented in the accompanying SHARP reports generally take
the form of hypothesis tests that aim to establish which of a null and an alternative
hypothesis had the greater probability of being correct. As an example, these could
take the following general form:
Null hypothesis: Intervention households, which moved to new-build
accommodation as part of the housing or regeneration programme,
experienced changes in circumstances or outcomes that were not
significantly different from those observed in Control households, which
were not rehoused as part of the programme.
Alternative hypothesis: Intervention households, which moved to newbuild accommodation as part of the housing or regeneration programme,
experienced changes in circumstances or outcomes that were
significantly different from those observed in Control households, which
were not rehoused as part of the programme.
6.11 “Change” in this context refers to change over time as measured in the
different waves of the project. It includes changes such as those in circumstances
(e.g., neighbourhood problems) and in health outcomes (e.g., improved SF-36 scores).
32
These include changes that may be regarded as improvement or deterioration. In
combination with the data from the near-simultaneously interviewed Control Group, it
was intended to be able to highlight overall changes in the general population that
might otherwise have been interpreted as being the consequences of the intervention.
6.12 The nature of the study allowed 2 analytical approaches, using the crosssectional and the longitudinal samples. These both have their particular advantages
and disadvantages. Frequently, the analyses were done with both samples, to
determine whether there were any striking differences in the findings of the analysis
that could have been ascribed to the systematic bias arising from the choice of sample.
6.13 The cross-sectional sample benefits from the inclusion of all respondents who
gave information at the Waves under analysis. Sample sizes are maximised in this
way with the consequence that it is easier, all other things being equal, to determine
statistical significance. However, comparisons between Waves or Intervention and
Control Groups (for example) involve differences in the samples as a whole, and do
not make full use of the information available about the changes experienced by
individual respondents within the Intervention/Control Groups over time. Although
the retention rate of respondents was very high in the SHARP project, we felt that it
was unwarranted to discard the substantial amount of information gathered from
respondents at Wave 1 simply because they had not participated, for whatever reason,
at Wave 3. Nevertheless, the ability to analyse longitudinal data, albeit from a smaller
sample, is the great design strength of the SHARP project, whereby we can examine
how aspects of the health and well-being of respondents and their households changed
over time following rehousing in new-build accommodation.
6.14 Categorical outcome variables (e.g., Likert scales, or dichotomous “yes/no”
type variables) were analysed using Pearson Chi-square tests for association in
contingency tables for bivariate analysis. To consider the effects of more than one
independent variable simultaneously, the analyses were extended to include loglinear
and logit modelling of frequencies and proportions, respectively. For some analyses
of the longitudinal sample, a derived variable of the differences in responses (Wave 3
- Wave 1) was analysed.
6.15 Continuous or pseudo-continuous variables were checked for significant
deviation from normality, transformed where necessary, and analysed with standard
parametric techniques. Thus, differences between group means were investigated by ttests or by ANOVA, the latter allowing the simultaneous consideration of the
differences between 2 or more independent variables (or of variables with more than 2
levels). For the analyses of the longitudinal sample, the corresponding repeated
measures t-tests and ANOVAs were employed to estimate separately the withinsubjects and between-subjects effects and interactions.
6.16 Strictly speaking, even though we formulate our hypotheses in such a way as
to assign implicitly the set of independent and dependent (outcome) variables under
investigation in any test, we should be clear that a statistically significant result tells
us nothing about the direction of causality of the relationship. As an example, let us
imagine that we have found an association between respondent age and the tendency
to live in houses rather than flats. While it is possible to rule out some causal
relationships on non-statistical grounds (e.g., the type of dwelling a person lives in
33
cannot affect their age), it is not possible to prove that the causality runs in the
opposite direction (that respondent age affects the type of dwelling in which they live)
on the basis of a test statistic alone.
6.17 The problem of interpreting statistical results, in common with all studies, is
compounded by the correlation that is likely to exist between our key explanatory
(independent) variables. While the effort required to gather high quality information
from respondents was considerable, it should be recognised that the maximum sample
size of 723 respondents available for any one analysis was too small to allow adequate
replication of all combinations of classes of even a small number of independent
variables. Thus, instead of carrying out subgroup analyses, we have often analysed
separately the same outcome variables with respect to key independent variables (e.g.,
Intervention vs. Control Group, relocator group, landlord and tenure change group,
SIP status group). Correlation between any of these is likely to lead to similar patterns
of significance being identified, which might lead to the inference of incorrect causal
explanations.
Investigating changes in outcomes
6.18 When analysing changes in our main outcome measures, such as variables
relating to mental health or physical health, we proceeded through a series of analyses
as detailed below. Sometimes this involved creating secondary, novel variables from
variables already analysed, for example, in studies of housing change and social
change.
Change over time
6.19 Changes in the outcome variable between Wave 1 and Wave 3 were compared
between the Intervention and Control Groups, looking for example at differences in
mean values, or at differences in the proportions of the 2 groups giving a particular
response to a categorical variable (e.g. the proportion with a long-standing illness).
Change by household type
6.20 The analysis was repeated within and between the 2 study groups, by dividing
the study groups (i.e. Intervention and Control Groups) into 4 household types, as
defined at Wave 1, namely: families with dependent children; families with nondependent children (aged 16 or over); adult households; and older person households.
Statistical tests were then carried out both to see if the outcome varied across
household types within each study group, and also to test whether the patterns in
outcomes by household type was any different between the 2 study groups: for
example, does the pattern of change in psychosocial benefits differ across household
types in a different way in the Intervention Group as compared with the Control
Group.
34
Change by dwelling transition
6.21 For the Intervention Group, we wished to see if the type of house move
involved in rehousing had any effect upon the outcomes, so we analysed the outcome
measures according to a classification of the dwelling transition experienced. We
identified 4 types of dwelling transition for this purpose, which were, in descending
order of incidence: flat to house (45.6%); flat to flat (26.6%); house to house
(20.1%); and house to flat (7.7%).
Aspects of dwelling change
6.22 Again, for the Intervention Group, we wished to know whether particular
aspects of dwelling change had any significant effects upon the outcome measures.
To do this, we used individuals’ responses at Waves 1 and 3 to measure aspects of
dwelling change within the longitudinal sample as follows:
Floor level
6.23 Four change groups were identified within the Intervention Group: higher to
ground (45.0%); ground to ground (38.2%); ground to higher floor (8.8%); and higher
to higher (8.0%).
Access to garden
6.24 Most people ended up with a garden for their own use by Wave 3. We
initially identified 4 change groups, but since one of them (garden to no garden)
contained few cases, we collapsed this for analysis purposes into 3 groups: no garden
to own garden (43.9%); own garden to own garden (34.4%); and, no access to own
garden at Wave 3 (21.8%).
6.25 Respondents were asked about 28 potential dwelling problems at Waves 1 and
3. We grouped these items into aspects of the dwelling, and used the changes in
people’s responses to these questions to categorise their degree of improvement. We
then used these simpler measures as independent variables in our analyses:
Dwelling fabric
6.26 From 5 items about the dwelling fabric, we used changes in the number of
items identified by respondents as ‘serious problems’ to classify people as follows:
best gain in fabric conditions, defined as a drop of 3 or more serious problems
(16.2%); less gain, defined as a drop of 1 or 2 serious problems (17.4%); no gain, or
loss, in fabric conditions (66.4%, of which only 1.6% is a loss or deterioration in
fabric conditions).
35
Safety
6.27 From 4 items about the dwelling safety, we used changes in the number of
items identified by respondents as ‘serious problems’ to classify people as follows:
gain in safety, defined as a drop of 1 or more ‘serious problems’ (23.5%); no gain or
loss in safety (76.5%, of which only 5.4% represented a deterioration in safety).
Amenities, warmth and comfort
6.28 From 4 items about amenities in the home (warmth, drying clothes, baths and
showers, and double glazing), we used changes in the number of items identified by
respondents as ‘serious problems’ to classify the Intervention Group as follows: best
gain in amenities, defined as a drop of 2 or more in the number of ‘serious problems’
(22.8%); less gain, defined as a drop of 1 serious problem (18.9%); and not gain or
loss of amenities (58.3%, of which only 5.8% represents a deterioration in amenities).
Dwelling space
6.29 Respondents were asked 4 questions about problems with room sizes and with
the number of rooms. However, since the questions were paired (too many rooms
versus too few rooms; and rooms too small or rooms too large), the actual number of
potential problems is only 2. Thus, we classified people on the basis of their
responses over time as follows: gain in space, defined as a drop of 1 or more serious
problems (30.9%); and no gain or loss of space, defined as no change or an increase in
the number of serious problems with space (69.1%, of which only 5.8% were
deteriorations in space, i.e. increase in the number of serious problems).
Privacy and quiet
6.30 Respondents were asked one question about privacy and 3 about sources of
noise. Using the responses to these questions, we classified the Intervention Group
respondents as follows: gain in privacy and quiet, defined as a drop of 1 or more
serious problems (31.9%, of which only 9.3% experienced a drop of more than 1
serious problem); and, no gain or loss of privacy, defined as no change or an increase
in the number of serious problems (68.1%, of which only 9.8% had a loss, i.e. an
increase in the number of serious problems).
Change by location status
6.31 By means of the Wave 2 postal questionnaire, we were able to classify the
Intervention Group according to whether or not they had moved area as well as house.
This resulted in the following classification across the 2 study groups:
Relocators: People who moved to a new-build dwelling in a new
neighbourhood, comprising 25.3% of the Wave 3 sample.
36
Movers: People who moved to a new-build dwelling in the same
neighbourhood where they previously lived, comprising 23.7% of
the Wave 3 sample.
Remainers: People who remained in the same dwelling over time,
comprising 51.0% of the Wave 3 sample.
Change in outcomes by change in neighbourhood conditions
6.32 Just as with dwellings, respondents were asked a series of 23 questions about
problems in their neighbourhoods. We grouped these into 2 domains as follows:
neighbourhood infrastructure, services and environment (12 items); and
neighbourhood crime and anti-social behaviour (11 items). For each domain,
respondents were divided into 3 groups: those with the best gain, defined as a drop of
3 or more serious problems (25.2% of the Intervention Group sample for the
infrastructure domain, and 26.7% for the crime domain); less gain, defined as a drop
of 1 or 2 serious problems (27.1%, and 22.4%, respectively); and no gain or loss
(47.7% and 51.0% respectively).
6.33 We use the SHARP sample to look at whether residential, social and health
outcomes differ according to whether or not respondents in both study groups are
living in a regeneration area or not. Regeneration areas at the time were called Social
Inclusion Partnership Areas (SIPs). We used the postcodes of our respondents
together with a postcode definition of SIP areas in order to locate our sample. Our
samples were fairly evenly divided between SIP areas and other areas at both main
survey waves, as shown in table 6.1.
Table 6.1
Study group samples by SIP status
Not in SIP
Wave 1
Intervention Group
Control Group
TOTAL
Wave 3
Intervention Group
Control Group
TOTAL
SIP
Total
164
49.1%
205
52.8%
369
51.1%
170
50.9%
183
47.2%
353
48.9%
334
100.0%
388
100.0%
722
100.0%
138
52.3%
158
55.8%
296
54.3%
124
47.3%
125
44.2%
249
45.7%
262
100.0%
283
100.0%
545
100.0%
37
Mental health outcomes in relation to social outcomes
6.34 We examined the relationships between social and community outcomes for
Intervention Group respondents on the one hand, and mental health and well-being
outcomes on the other. This tested for the possibility that rehousing had indirect
effects upon mental health through its effects upon people’s social networks and
perceptions of community. A number of measures of social outcomes were utilised
as described below.
Use of local amenities
6.35 Respondents were asked at Wave 1 and Wave 3 which of a list of amenities
they made use of in their local area. We looked at change in these responses in order
to classify people according to whether they used more local amenities at Wave 3
(40.1% of respondents); had no change in the number of local amenities used
(21.4%), or used fewer amenities at Wave 3 (38.6%).
Local participation
6.36 Similarly, respondents were asked whether they participated in a range of local
clubs and organisations, from which we produced a count at both Wave 1 and Wave
3. We then classified Intervention Group respondents according to whether they
participated more at Wave 3 (26.0%); participated in the same number of groups at
both Waves (43.9%); or had reduced their participation (30.2%).
Established local social network
6.37 Respondents were asked how many close family members and close friends
they had living nearby, at both points in time. A change measure was constructed that
divided people into those who had more local friends and family at Wave 3 (28.5%);
no change (19.6%); and those who had fewer local family and friends after moving
house (51.9%).
Neighbouring behaviours
6.38 From a set of questions about interactions with neighbours (such as visiting
neighbours in their homes), Intervention Group respondents were grouped according
to whether they engaged in more neighbouring behaviours after moving (45.0%); saw
no change in their neighbouring behaviours (24.8%); or reduced their neighbouring
behaviours (30.2%).
Sense of community
6.39 The module of questions about people’s communities allowed us to construct
a series of measures of different aspects of community at both Wave 1 and Wave 3:
38
these variables were numerical scores produced by combining the responses to
different groups of questions covering people’s sense of belonging, cohesion,
empowerment, safety and collective efficacy. By looking at the change in people’s
scores, Intervention Group respondents were classified as shown in table 6.2.
Table 6.2
Measures of change between Wave 1 and Wave 3 in sense of
community in the Intervention Group (figures show change in
percentages)
Increase
No Change
Reduction
(Row Pct)
Belonging
48.9
19.6
31.5
100.0
Cohesion
55.0
45.0
100.0
Empowerment
57.3
42.8
100.0
Safety
52.7
24.9
22.5
100.0
Collective Efficacy
51.5
48.5
100.0
Note to table: Where there were very few ‘no change’ cases, they have been combined with the
‘reduction’ category, to allow enough numbers for a comparison with the “Increase” category.
39
Summary
•
The management of the dataset included checking the electronic data for 5%
of the cases against the paper copies of the questionnaire for quality control
purposes. As a result of quality checks, 1.5% of the data cells for the
Intervention Group and 2.5% for the Control Group were amended to correct
systematic errors, mostly related to a small number of questions with
complicated response categories.
•
Analysis of the data involved using both the cross-sectional samples, to
maximise the use of data and the determination of statistical significance; and
using the smaller longitudinal sample to avoid biases introduced by attrition
and to get a true picture of change over time on an individual basis.
•
Statistical test employed in the analysis included Pearson Chi-Square tests for
categorical variables, and t-tests and ANOVA for continuous or pseudocontinuous variables. To control for the effects of more than one variable, log
linear and logit modelling was used for variables in the form of frequencies
and proportions, respectively.
•
The analyses involved looking at changes over time, comparing the
Intervention and Control Group, as well as changes experienced by different
types of household, looking at differences in these household experiences both
within and between the Intervention and Control Groups.
•
Thereafter, a series of analyses were conducted on the Intervention Group to
look at various influences upon their outcomes. First, this involved examining
the nature of the dwelling transition made through rehousing, followed by
examining the effects of different aspects of dwelling change (e.g.
improvements in dwelling fabric or space). Second, the effects of location
were studied by looking at whether or not people had moved area, whether or
not they lived in regeneration areas, and at whether 2 key dimensions of the
neighbourhood had changed (the environment and services, and crime and
anti-social behaviour).
•
Lastly, we constructed summary measures from the social outcomes to
examine their influence upon respondents’ mental health and well-being
outcomes.
40
CHAPTER SEVEN
QUALITATIVE RESEARCH
Aims of the qualitative research
7.1
Two waves of in-depth qualitative interviews were conducted with subsamples of SHARP survey respondents. The first wave was conducted between one
and 3 years after respondents had moved into their new homes. The second wave was
conducted between 3½ and 5 years after respondents had moved. The aims were
similar in each wave, that is to explore the impacts of housing and area change on a
range of health, community and social outcomes from the perspectives of the
respondents. Specifically, the first wave aimed to:
• Highlight issues to pursue in the quantitative analysis;
• Examine how respondents talk about housing and health; and
• Focus on the impacts of change and disruption on the lives of respondents.
7.2
The second wave was concerned with investigating the impact of moving into
new-build social housing and moving to a new area or living in a regeneration area
on:
• Housing conditions and satisfaction with new housing;
• Physical health, health behaviours and well-being;
• Community and social outcomes such as neighbouring behaviour and sense of
community; and
• Area outcomes such as anti-social behaviour and visual amenity.
7.3
The methodological approach adopted for each qualitative wave is described
in the following sections.
Post-Wave 2 qualitative research
Sampling and recruitment
7.4
Twenty-eight respondents were interviewed in the first qualitative wave. They
were selected using a quota sampling system designed to include respondents who
had lived in their new accommodation for varying lengths of time, in order to explore
potential differences in their experience. The sample also aimed to include
approximately equal numbers of respondents who lived in SIP and non-SIP areas, and
to interview 5 respondents who had moved to rural areas. No specific quotas in
respect of gender were sought.
7.5
Respondents were contacted by mail in the first instance, prior to being
telephoned to request their participation in the research. All of those who were
contacted by telephone agreed to take part. The characteristics of the achieved sample
are set out in Table 7.1 below. Eighteen women and 10 men were interviewed.
Thirteen interviews were conducted with respondents from ‘family’ households, 6
with those from ‘adult’ households, and 9 with those from ‘older person’ households.
41
Table 7.1: First wave qualitative sample characteristics
No.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26*
Interval:
moving interview
12 months
12 months
12 months
12 months
12 months
13 months
13 months
13 months
13 months
15 months
16 months
16 months
15 months
13 months
14 months
12 months
14 months
21 months
17 months
20 months
30 months
25 months
19 months
33 months
34 months
Location
Rural/ Urban
Designation
SIP/
NonSIP
NIS
NIS
NIS
NIS
NIS
NIS
SIP
SIP
SIP
NIS
NIS
NIS
NIS
NIS
NIS
NIS
NIS
SIP
SIP
SIP
SIP
SIP
D/K
SIP
SIP
SIP
Sex
Age
Children
Paisley
Large urban area
M
52
0
Whitburn
Other urban area
F
28
2
Whitburn
Other urban area
M
35
0
Perth
Other urban area
M
79
0
Paisley
Large urban area
F
30
2
Argyll and Bute
Remote rural
F
39
1
Glasgow
Large urban area
F
42
2
Glasgow
Large urban area
M
63
2
Glasgow
Large urban area
F
38
4
Perth
Other urban area
M
52
0
Perth
Other urban area
F
65
0
Perth
Other urban area
F
72
0
Perth
Other urban area
F
73
0
Stirlingshire
Accessible rural
F
22
2
Stirlingshire
Accessible rural
F
29
1
Stirlingshire
Accessible rural
F
58
0
Stirlingshire
Accessible rural
F
27
3
Glasgow
Large urban area
F
68
0
Glasgow
Large urban area
F
57
1
Glasgow
Large urban area
M
44
0
Dundee
Large urban area
M
71
0
Dundee
Large urban area
M
57
2
Glasgow
Large urban area
F
30
1
Glasgow
Large urban area
M
49
1
Dundee
Large urban area
F
50
0
West
Rural accessible
F
D/K
D/K
Dunbartonshire
27*
West
Rural accessible
SIP
M
D/K
D/K
Dunbartonshire
28*
West
Rural accessible
SIP
F
D/K
D/K
Dunbartonshire
Note to table: * These pilot interviewees were originally from a related study of housing
improvement and the information for this survey does not always coincide exactly with that for
SHARP.
Interview schedule
7.6
The interview schedule was developed by the research team with the advice of
colleagues. It was designed to investigate: general aspects of the respondents’ new
homes; differences between new homes and previous accommodation; relationships
with neighbours and the wider community; the health of the respondent and their
family, and respondents’ attachment to the area in which they now lived.
42
Conduct of interviews
7.7
The interviews were conducted in the respondents’ own homes. In most cases,
they were one-to-one interviews at which only the respondent was present. However,
on a few occasions, the partner of the respondent was also present and contributed to
the interview. Informed consent was obtained prior to the interview commencing,
following standard procedures for such research. Subject to obtaining consent from
the respondent, interviews were audio taped.
Analysis
7.8
The interview tapes were transcribed by the interviewer or by a professional
transcription company. The transcripts were read and analysed by 2 researchers, who
identified the key subjects raised by the questions in the interview schedule.
Post-Wave 3 qualitative research
Sampling and recruitment
7.9
A purposive approach to sampling was adopted, wherein study participants
were selected on the basis of specific criteria of interest. The findings of the first
wave of qualitative interviews suggested that the impact of moving was mediated by
age, household composition and distance of move. The intention was therefore to
interview approximately equal numbers of respondents from each household type (i.e.
Adult, Family and Older) who had either moved to a new area or stayed in the same
area. The focus was on urban respondents resident in the Greater Glasgow area.
Thus, the sample framework set out below was devised:
Table 7.2: Qualitative sample target framework
Characteristics
Adult no children
Family
Elderly
TOTAL
Same area
4
4
4
12
New area
4
4
4
12
Total
8
8
8
24
7.10 Respondents were initially contacted by letter, requesting permission to
conduct an interview. If the enclosed reply slip was not returned within 2 weeks, and
there was an active phone number available, they were then contacted by telephone to
request permission. Since some difficulty was encountered in attaining the original
sample framework, the original target was amended slightly by reducing the number
of older respondents in the sample to 6. Thus, there were 22 respondents in the final
achieved sample. It was also necessary to expand the geographical area initially
targeted, to include non-urban areas and locations outwith Glasgow20.
7.11 Of the 22 respondents, 19 were female and 3 were male. The majority were
resident in the Greater Glasgow area, including Faifley, Paisley and Cumbernauld.
20
In the event, 2 of the 22 respondents lived outwith the Glasgow conurbation.
43
One respondent was resident in Whitburn, and one in Blantyre. The respondents had
lived in their current accommodation for between 3½ and 5 years. The sociodemographic characteristics of the sample are presented in Table 7.3 below.
Table 7.3
Study ID
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
Sociodemographic characteristics of the sample
Age
33
33
38
46
46
48
49
50
51
53
54
55
55
56
56
60
64
66
72
78
78
81
Gender
Male
Female
Female
Female
Female
Female
Female
Female
Female
Female
Male
Female
Female
Female
Female
Female
Female
Female
Female
Male
Female
Female
Employment Status
FT
PT
Housewife
FT
FT
Long-term sick/disabled
FT
FT
Long-term sick/disabled
Long-term sick/disabled
Long-term sick/disabled
PT
Long-term sick/disabled
Long-term sick/disabled
PT
Long-term sick/disabled
PT
Retired
Retired
Retired
Retired
Retired
HH type
Family
Family
Family
Adult
Adult
Adult
Family
Family
Family
Adult
Adult
Adult
Family
Family
Adult
Adult
Older
Older
Older
Older
Older
Older
Interview schedule
7.12 The interview schedule aimed to capture changes in the broad areas of housing
and area change and attendant impacts on health, well-being, social and community
outcomes.
Conduct of interviews
7.13 One-to-one in-depth interviews were conducted between August 2007
January 2008. The interviews were conducted in respondents’ homes, except
which was conducted at the respondent’s place of work. Informed consent
obtained from the respondents, and a shopping voucher to the value of £10
provided to thank respondents for their time.
and
one
was
was
7.14 In the majority of cases, only the respondent was present during the interview.
However, on 3 occasions the respondent’s partner was an active contributor. Subject
to the participant’s permission, the interviews were recorded using digital recording
equipment.
44
Analysis
7.15 The interviews were transcribed by a professional transcription company in
preparation for analysis. The software package NVivo 7 was used to conduct a
thematic analysis of the data. This involved identifying the overarching themes of
interest in the research context, then coding text in the interview transcripts which
corresponded to these themes. The coded text was then examined in detail to identify
recurring themes and patterns within the data, which were then sub-coded in a further
round of analysis. When this process was completed, connections between emergent
themes and respondent characteristics were investigated in some depth.
Limitations
7.16 As with all qualitative research, the primary limitation of this element of the
SHARP study is that the findings cannot be generalised to the wider population. A
strength of the SHARP study, however, is that the mixed method approach allows
triangulation between the quantitative and qualitative findings. There are also a
number of limitations specific to the second qualitative wave. The method of
recruiting respondents from an existing survey sample may lead to selection bias, in
that individuals with particular characteristics may be more likely to agree to
participate in a further wave of the research. Further, respondents with certain
characteristics form the majority of the sample: for instance, those who moved from a
flat to a house and those who live in a regeneration area predominate within the
sample. Although this is not dissimilar to the quantitative sample distribution, it
means that the absolute numbers with certain characteristics in the qualitative sample
are so small that conclusions based on the findings for particular groups are somewhat
tentative. It should also be noted that there is a gender imbalance, with only 3 men in
the sample.
Summary
•
Two waves of qualitative interviews aimed to explore experiences of moving
and living in new-build social housing from the perspectives of the
respondents.
•
One-to-one in-depth interviews were conducted shortly after Waves 2 and 3 of
the survey with sub-samples of survey respondents.
•
The interview schedules were designed to investigate the impact of moving on
respondents’ housing conditions, health, and social and community outcomes.
•
The interviews were recorded and transcribed for analysis. Thematic analyses
focussed on themes pertaining to the research questions.
•
The findings are not generalisable to the wider population. However,
triangulation of the qualitative and quantitative findings permits further
exploration of similar and conflicting findings.
45
CHAPTER EIGHT
STUDY STRENGTHS AND LIMITATIONS
8.1
There is much ongoing debate about the lack of public health evidence and the
need for more evidence of the effectiveness of public health interventions, and
upstream interventions such as housing improvement. It is widely acknowledged that
there are many barriers to this type of research, and it is very rare: it has been
estimated that a remarkably low 0.4% of academic and research output is relevant to
public health intervention research21. SHARP is one of these studies and should be
seen in this context. We have outlined below a selection of the major strengths and
limitations of SHARP.
Strengths of the study
Prospective controlled design
8.2
We feel that one of the main strengths of SHARP is that it is one of the
uncommon 0.4% - a prospective, controlled study of the health and other impacts of
new social housing. Its controlled design is in itself a strength, allowing us to
differentiate between natural changes over time in individuals from changes which
can be attributed to the change in housing and other circumstances, including in some
cases, changes in the neighbourhood. This is important because it has been shown
elsewhere that the lack of a controlled design may result in over-estimation of the
effects of the intervention.22
8.3
The length of follow-up compared to other studies is both a strength and a
potential weakness; while few previous prospective studies have followed
respondents up for longer than a year – SHARP followed respondents for 2 years – it
is likely that some clinically-important changes in health will only be detectable over
much longer periods. This may be why rehousing did not result in noticeable gains in
physical functioning in the Intervention Group, and there was little change in
symptom reporting. Ideally, one would continue to follow the sample over 5 or more
years, but in practice this is difficult given attrition in the study population over time,
and given the likelihood of further house moves and loss of contact with respondents;
by 5 years the study numbers would probably be too small to permit meaningful
quantitative analysis.
8.4
We should note here also that the response rates which were achieved in
SHARP are very high compared to similar studies conducted in deprived areas. It is
commonly reported that survey response rates in such areas are in decline. Response
rates to postal surveys in particular are often very low, and now lucky to achieve 2030%. In that light our response rate at Wave 2 of over 80% is remarkable. The fact
that even at Wave 3 – 2 years later, we retained 78% of our original sample in the
study is also a significant achievement and a testament to both the hard work of the
survey team and the dedication of the SHARP interviewers, but also to the real
21
http://www.nice.org.uk/niceMedia/documents/pubhealth_intervention.pdf
Cummins SCJ et al. Large-scale food retailing as intervention for diet and health: quasi-experimental
evaluation of a natural experiment. J Epidemiol Community Health 2005; 59: 1035 - 1040
22
46
engagement with the study which the SHARP respondents clearly felt. The high
response rate may be related to the type of respondent in the SHARP study, who are
more likely to be economically active than those living in similar housing elsewhere
in Scotland – approximately 38% of SHARP respondents were working at Wave 1
compared to 17% of Glasgow Housing Association tenants in 2004.
Multi-site study
8.5
Most previous studies of housing and health have been focused on a single
housing development or an improvement programme located in one particular place.
This has limited their generalisability. SHARP overcomes this problem by selecting a
sample of developments for study from across the geographical breadth of Scotland.
As such, it is a programme evaluation which can produce generalisable findings,
balancing effects in one place against those in another. The findings are not therefore
subject to the particularities of place (whether of situation or of resident group).
Holistic view of the outcomes of housing investment
8.6
We feel that one of the key strengths of the study is that it takes a genuinely
holistic view of the outcomes of housing change. Previous studies have tended to
include a small selection of (mostly health-related) outcomes. However in SHARP we
did not assume that health was the only outcome of interest, but recorded data on a
wide range of social outcomes, reflecting the similarly broad goals of housing
improvement and regeneration policies. Thus we were able to note for example
change in community cohesion over time in regeneration areas.
Exploring processes and mechanisms
8.7
When SHARP was designed initially we started by outlining the pathways by
which housing and neighbourhood change may lead to changes in health and other
outcomes. This means that we have an unusually rich set of data describing the
processes and mechanisms by which change takes place – including detailed
qualitative data. We have used these data for example to explore how one’s sense of
security and of community cohesion relate to changes in housing and neighbourhood
characteristics, but other more detailed analyses are also possible. This increases
confidence in the conclusions; where we are able to show not just a change in a
behaviour, but also the other alterations in circumstances which accompany that
change, then it is more likely that the observed effect is real.
Study limitations
Size and Duration
8.8
The limitations of study size and duration are already described above.
Although SHARP is larger than most previous studies (with some exceptions, most
previous controlled studies have been very small – often with fewer than a hundred
47
participants in each group) the numbers are still relatively small, which affects the
study’s ability to detect rare events, and in particular limits our ability to explore
subgroup effects.
Control Group
8.9
The major weakness however is the lack of an identical Control Group. This
could realistically only have been achieved by means of randomisation which was not
possible for reasons given in earlier chapters. Only randomisation (with a large
enough sample) could have ensured that the Control and Intervention Groups were
comparable with respect to all known and unknown confounders. Despite the
matching at baseline, and subsequent statistical adjustment for selected possible
confounders, there was an imbalance between the groups in several key variables at
baseline (e.g., accommodation type), and there was probably an imbalance in some
unknown ones, as with any observational study. In general, such imbalances do not
cancel out, but tend to result in over-inflated estimates of the effect of the intervention
on outcomes – for example, housing improvement would be found to be more
effective in improving health than it really is. The size of such inflation is impossible
to quantify, but such a caveat needs to be borne in mind when interpreting the results.
Conclusion
8.10 Nevertheless we are confident that SHARP will be seen as a innovative study
which contributes to a greater understanding of the relationships between housing
improvement, neighbourhood change, and the social and other changes which flow
from these activities. It provides important pointers for future policy development and
provides a strong basis for future research in this field. It also provides a reminder that
health improvement is not the most important basis for social housing improvement
and regeneration, and a reminder that evaluation of complex social interventions
requires complex, multidisciplinary social research, which acknowledges the complex
and inter-linked nature of the outcomes.
48
Summary
The main weaknesses of SHARP include:
•
The fact that it is not a randomised experimental design;
•
Difficulties with obtaining identical Intervention and Control Groups at
baseline; and
•
The study size, which though larger than many other studies, precludes
detailed subgroup analysis.
The main strengths of the SHARP study are:
•
Its prospective controlled design, albeit with limitations;
•
Its multidisciplinary nature;
•
It is a multi-site study, spread across a range of locations throughout Scotland;
•
It has high retention and low attrition rates;
•
The multi-dimensional outcome assessment, including health and social
processes and outcomes; and
•
The fact that SHARP investigates housing in the context of neighbourhood
change, which few studies have done – thus allowing the investigation of the
synergy that may take place when housing improvement takes place in the
context of wider regeneration programmes.
49
ISSN 0950 2254
ISBN 978-0-7559-7283-8
(Web only publication)
www.scotland.gov.uk/socialresearch
The text pages of this document are produced from 100% Elemental
Chlorine-Free material.
The paper carries the Nordic Ecolabel for low emissions during
production, and is 100% recyclable.
RR Donnelley B58284 10-08