Evaluating Services & Expenditure in Social Sectors
Approaches supported by The Atlantic Philanthropies
Gail Birkbeck
Feb 1, 2013
Overview
o Introduction to The Atlantic Philanthropies
o Evaluation at Atlantic
o Some examples
2
About Atlantic
o Atlantic has made grants to advance opportunity and lasting change for
those who are unfairly disadvantaged or vulnerable to life's circumstances.
o A limited life foundation, we make grants through our Ageing, Children &
Youth, Population Health and Reconciliation & Human Rights Programmes.
o We are active in Bermuda, Northern Ireland, Republic of Ireland, South
Africa, the United States and Viet Nam.
o We have made grants totalling over $6.1billion
o At the heart of our work is the belief that all people have the right to
opportunity, equity and dignity.
3
Evaluation and Learning - Purpose
o Informing strategy and implementation
o Assessing impact and context
o To enhance capacity, facilitate learning and share knowledge
4
Evaluation Design and Approaches
o Grounded in questions critical to understanding effects and impact
o A ranges of methods are used
o Different level of analyses – individual, community, programme,
Atlantic, philanthropy
o Some approaches are more helpful in the early development
stage, others more suited to ongoing programmes, others still
where information is needed quickly
5
Children & Youth Programme
Promote Prevention & early intervention
o Demonstrate effective practice leading to policy reform
o Inform and influence policy and practice
o Develop capacity & infrastructure for the sector
International trends towards increasing emphasis on prevention strategies to
cost-effectively address social problems
6
April 6, 2015
April 6, 2015
9
April 6, 2015
April 6, 2015
Evidence-based Service Design
o Selecting services based on the evidence of specific community
needs
o Selecting services models or interventions with evidence of
effectiveness
o Testing their effectiveness in local communities
12
Incredible Years - RCT of Teacher Classroom
Management and Parent Training programmes.
o Significantly improves teachers’ competencies and their
management of disruptive behaviours in the classroom.
o Benefits pupil behaviour and socio-emotional adjustment.
o Reductions in child conduct disordered and hyperactive-type
behaviour.
o In total, 71 per cent of children showed improvements in behaviour.
o Parents reported reduced distress and improved well-being.
o Substantial decline at follow-up in the use of primary care and social
work services in the intervention group
13
Service Utilisation
Proportion (%) using health and social care services at baseline and follow-up
Sample (n=103)
Service
Baseline
6 month follow-up
12 month follow-up
GP
65
49
40
Nurse
8
5
2
Speech Therapist
24
15
9
Physiotherapist
6
2
3
Social Worker
10
1
2
Community paediatrician
5
3
0
SNA
11
11
13
Casualty department (A & E)
15
14
11
Outpatient consultant appointment
22
14
16
Overnight stay in hospital
7
8
6
Positive changes in child and parenting behaviour were
maintained at followup in the context of reduced support from
primary health care and social care services.
12 Month Follow-On Report (IY Parent Teacher Programmes)
McGilloway et al. 2012
14
Cost Analysis
Calculating Costs of Individual Services
o Unit costs for key services identified in the SUQ was collected
o Cost of service use at the 12-month follow-up (€626.91) amongst
the intervention group was reduced to 60% of baseline
expenditure on formal services (€1047.91).
Cost per child at baseline and follow-up (ITT)
o Primary Care
€158.55 - €68.61
o Hospital Services €453.50 - €166.18
o Special Education €428.13 - €386.36
A cost-effectiveness analysis of the Incredible Years Parenting
Programme in Reducing Health Inequalities ; O’Neill et al (2013)
15
Accessing Cost Data
o Ireland – Comprehensive Review of Expenditure (DCYA, 2011),
detail on expenditure but does not account for Health expenditure
o US - Urban Institute releases an annual report on child
expenditure
o 2011 federal expenditures rose overall but fell for children from
10 to 8%
o UK - LSE reports on public expenditure on children in the UK 'The way in which budgets are currently constructed across the UK
makes it very difficult to identify what is actually spent on children…‘
o Trends - education, social security & social services
Spending is being tracked although not connected to outcomes
16
Ageing Programme
Palliative Care
o A systematic review of palliative care
o Patterns & Costs of deaths in Ireland
o Cost and Cost effectiveness of alternative models of palliative care
CARDI – Centre for Ageing Research & Development in Ireland
TILDA
17
In conclusion
o Track record of evidence based service design
o Outcome and economic data in social services
o Communication of results
o Evaluation findings influenced our programmes
o The findings and lessons from our work will be one of our most
tangible legacies
18