Evaluating and Developing
the Early Education Pilot
for Two Year Olds
Evaluating and Developing
the Early
Education
Pilot
Gail Peachey
and Suzanne
Lunn
for Conference
Two Year Olds
DCSF Research
2009
Gail Peachey and Frances Miller
DCSF Research Conference
2009
Overview
– Acknowledgements
– Background
– Methodology
– Key messages
– Implementation
Acknowledgements
– Contractors:
• NatCen: Ruth Smith, Susan Purdon, Ivana La Valle
and Caroline Bryson
• University of Oxford: Sandra Mathers and Kathy
Sylva
• University of London: Eva Llloyd
– DCSF Research Team
– Policy: Suzanne Lunn, Frances Miller,
Background
• Policy
o Between 2006 and 2008 pilot provided free early years
education for over 13,500 disadvantaged two year olds
across 32 local authorities
o Available in a variety of settings
Aim of policy was to:
o improve children’s social and cognitive outcomes,
e.g. social confidence and independence and their
verbal and reasoning skills
o have a positive impact on parents and family, e.g.
parent-child relationship; parents’ emotional wellbeing
Background (cont’d)
• Evaluation design
o To assess whether or not the policy had
achieved its aims a programme of evaluation
was developed including:
o Mapping study looking at implementation and
targeting
o Quality assessments of pilot settings
o Impact study using matched comparison group to
measure impact on pilot children and families
o Qualitative interviews exploring parents’
experiences of the pilot and perceptions of impact
Mapping Study
• Why?
o Local Authorities were given the flexibility to implement the
pilot in a way they thought would best tackle local issues this
raised problems for an overall pilot evaluation
• Aim of the Mapping Study
o Gather detailed information on local implementation
strategies to inform the development of the evaluation
framework
• Methodology
o Contacted all 32 local authorities
o 45 minute interviews conducted with key local authority staff
o Issues covered included:
o How they chose their target groups
o Types of settings offering the pilot
o How the free hours were being delivered in their local authority
Quality Assessments
• Why?
o Quality of childcare is associated with differential child outcomes
and the quality of care offered by different settings could influence
the impact of the pilot
• Aim of the Quality Assessments
o To collect information on the quality of the different settings offering
the pilot
o Link setting quality information to individual children participating in
the impact study
• Methodology
o Collected setting characteristic information from all participating
settings
o Carried out observations at 75 settings that displayed a range of
characteristics (14% of settings; covered 38% of impact study
sample)
o Observations carried out using tried and tested rating scales:
o Infant-Toddler Environment Rating Scales (ITERS-R)
o Early Childhood Environment Rating Scale (ECERS-R)
o Caregiver Interaction Scale (CIS)
Impact Study
• Objective
– To support the future development of the two year old offer it
was important to understand exactly what impact the pilot
had had on the participating children and their families
• Methodology
– A quantitative, longitudinal, matched comparison study was
undertaken
• Compared results for the pilot group versus a
comparison group from other deprived areas of England
• Undertook face to face baseline interviews when the
child was aged 2 and in pilot areas close to when the
child took up their pilot place
• Undertook face to face follow-up interviews at age 3
• Cases were matched across a wide range of baseline
characteristics including child cognitive development at
age two
Impact Study (cont’d)
Sampling
• Pilot Areas
– Local authorities collected the details of all parents.
– Details of those who had not opted out of the evaluation
were passed to NatCen.
– Baseline: All 2,186 eligible parents were followed up
resulting in 1,400 interviews.
– Follow-up: 1,386 followed up; 1,116 completed interviews.
• Comparison Areas
– The comparison sample was selected from Child Benefit
Records with a skew towards non-pilot disadvantaged areas.
– Baseline: Following an opt out exercise 2,782 records were
issued and 1,821 interviews were conducted.
– Follow-up: 1,748 followed up; 1,376 completed interviews
Impact Study (cont’d)
Matching
• Propensity score matching was used to make sure that the pilot
and comparison samples are as similar as possible at baseline.
• Achieved by modelling the difference between the two samples
on a wide range of variables using logistic regression and
recording the likelihood of each person being in the pilot group.
• The treatment group individuals are then matched to the
comparison group individuals so that the final matched samples
have similar propensity score profiles i.e. they are equally likely
to be in the treatment group.
• The types of variables in the analysis included:
– cognitive social behaviour e.g. size of English vocabulary, level of
parental concern about how the child is learning and growing up
– Other variables e.g. use of childcare, receipt of income support,
degree of TV watching, lone parenthood
Qualitative Study
• 54 qualitative interviews were carried out in
January to February 2009
• Interviews lasted between 60 and 90 minutes
• Conducted with a range of parents (mainly
mothers) to explore:
– influences on take up
– perceptions of the impact of the pilot on children,
parents and families.
• Data was interpreted using a thematic
approach developed by NatCen.
• Provided insight to the broader impact of the
pilot.
Key Messages
•
•
The final report drawing together the key findings from
the study published in July 2009 concluded:
Implementation
o The pilot successfully targeted children experiencing different types of
disadvantage.
o However, the study found that around half of the control group were experiencing
childcare by the end of the pilot clearly raising questions about deadweight. How
many of the children in the treatment group would have experienced childcare
without the pilot?
o To reach the key target, the most economically disadvantaged children who, as a
group were less likely to be able to access good quality childcare, local authorities
would need to receive clearer guidance on targeting.
•
Impact
o To have a positive impact on language ability and parent-child relationships
children would have to attend higher quality settings – Ofsted rated as ‘Good’
•
Quality of settings
o The quality of provision in target areas needs to be improved – the majority of
provision in the pilot was ‘adequate’ (77%); only one fifth was ‘good’ (21%)
•
Parents’ experiences and perceptions of impact
o Overall positive: staff approachable and friendly, high satisfaction with feedback
on child’s progress, positive effect on child’s development
o Positive impact on parenting skills, physical health and mental wellbeing
Policy Development - Wave 1 LAs
• Original pilot
o In 2006, 32 LAs start delivering a targeted offer of
free early education to disadvantaged 2 year olds
(No cap on numbers – at LAs discretion)
o LAs took individual approaches in delivering the
offer.
o Definition of “Disadvantage” varied across the pilot
authorities.
o Different experiences of Quality in settings.
• Impact
o Findings from the evaluation influenced future
policy development bringing changes in the
implementation and delivery of the offer across the
Wave 2 (31 LAs) and Wave 3 (89 LAs)
Wave 2 Development
• Changes
o Instituted high level National eligibility criteria based on economic
disadvantage.
o Equalising the offer more – giving a harder focus on low income
families.
o Set delivery target – offer to reach15% of the most deprived
families in each Local Authority identified using the IDACI (the
Income Deprivation Affecting Children Index ).
o Wave 1 LAs working to bring numbers down and some to
increase numbers, to ensure offer reaches the 15% target group
o The offer is increased from 7.5 hours to 15 hours per child per
week over 38 weeks of the year across the Wave 1 & 2 LAs
o Quality criteria is reviewed and given a stronger focus
Policy Changes (Contd)
o
To be eligible for the offer all families must meet the primary
criteria - be in receipt of at least one of the following:
• Income Support
• Income-based Jobseeker's Allowance
• Child Tax Credit at a rate higher than the family element
• Extra Working Tax Credit relating to a disability
• Pension Credit
o
Local Authorities may also apply secondary criteria to meet
local priorities:
•
o
SEN / Teenage Parents / Lone Parents etc
Specific funding is issued to all LAs for Outreach / Family
Support Services targeting the hardest-to-reach families, and
those who will benefit most from an offer of free childcare.
Wave 3 and the National Offer
o Wave 3
o The Prime Minister wanted to move the offer further towards
being a universal offer for all two year olds
o Commitment to roll an offer of free early learning and childcare
“stage by stage” to all two year olds across the country –
starting with the most disadvantaged.
o Quality
o The final evaluation report highlighted a key component for the
offer to impact on child outcomes required all provision to be
of good quality
o Quality section in the Guidance is strengthened further (July
2009) to help LAs inform their local quality arrangements:
o Settings must have an Ofsted ‘Good’ or ‘Outstanding’
rating
o LAs may only use settings with a ‘Satisfactory’ rating if
they can evidence they are actively working towards a
good rating - to ensure sufficiency and standards to
deliver the offer
Implementation
o DCSF has provided guidance to LAs and support via Government
Offices, to run regional learning and knowledge sharing
workshops with LAs to share good practice and drive delivery
o September 2009 saw the remaining 89 (Wave 3) LAs start
delivery of the offer
o Wave 3 LAs are offering 10 hours per child, per week over
38 weeks of the year
o Currently providing funding for more than 20,000 children
per year and their families.
o The extended offer will be data-driven to ensure effective
targeting of the hardest-to-reach families, and those who
will benefit most from an offer of free childcare.
o All 152 local authorities across the country have been
funded to deliver a targeted offer to 15% of their most
disadvantaged 2 year olds by 2010-11
Next Stages
o Secure take up of the number of places allocated per Local
Authority; identify and share good practice in local delivery
o Further extend the number of places for the most disadvantaged
2 year-olds beyond 2011
o Improve access amongst Families with SEN
o Look further at what high quality provision for two year olds
means in practice - parents perceptions of quality childcare/early
years education and its broader impact
http://www.dcsf.gov.uk/everychildmatters/earlyyears/
localauthorities/lapractice/pilots/twoyearoldsoffer/