Educational Psychology in Scotland
Annual Conference
Herriot Watt University
September 14th 2012
The Preschool PATHS Curriculum:
Using Implementation Science to Increase
Effectiveness
Dr Barbara Kelly, Carole Edgerton,
Elaine Robertson, Diann Neil
Glasgow South Psychological
Services
1
Outline of Presentation
•
•
•
•
•
What is PATHS?
Preschool PATHS project in Glasgow South
Focus on Implementation Science
Interim Audit of IMPLEMENTATION
Glasgow PATHS Network Building Sustainable
Programmes
• Fuller discussion and references for all the issues raised
can be found in Barbara Kelly and Danny Perkins 2012,
Cambridge Handbook of Implementation Science for
Psychology in Education
2
PATHS Curriculum in Glasgow
• Fairer Scotland Preschool PATHS project one of few
•
•
•
externally funded in Glasgow`s education context
PATHS primary curriculum is becoming well known
throughout Scotland and has a developing evidence base
locally helping to fulfil Glasgow`s vision for children's
well being and welfare
Preschool PATHS is a fairly recent development in
Glasgow and is currently being embedded in a number
of preschool establishments
Preschool project focuses on Implementation Science
with a view to increasing sustainability and maximising
effectiveness
3
Why choose PATHS and
Preschool PATHS?
• Recognised as one of only eleven `blueprint` programmes
•
•
•
•
recommended for reducing adolescent crime, aggression and
substance abuse in the US (Mihalic et al 2001)
Significant positive outcomes for ASN pupils and in mental health
contexts ( Greenberg et al, 2000)
Study in England demonstrated greater cooperation, empathy and
self control (Curtis and Norgate, 2007)
In Scotland enhanced pupils` ability to manage difficult emotion and
positive impact on behaviour ( Kelly et al, 2004)
Key paper for Preschool PATHS effectiveness Domitrovich et al 2009
Reflects government priorities for developing
children`s mental health and well being
4
Why choose PATHS?
• Evidence of impact in primary and preschool
•
•
PATHS is building globally
Underlying dynamic, theoretical framework
proving to be more and more relevant to our
understanding of child development and learning
The central concepts of integration of
functioning and the importance of selfregulation have anticipated research findings
and are key emerging theoretical models in
developmental psychology
5
PATHS Impacts
• Offers a range of strategies to recognise
•
•
manage and regulate feelings and develop
alternatives to hostility and violence
Impacts on attainments –literacy and language
Linked closely to cognitive skills and problem
solving: emotional understanding underpins
effective thinking
But emotional education may have wider effects: literature
from theory of mind research and developmental
psychology suggests that emotion content of
conversations between parents and children creates
feeling, thinking and behavioural templates
6
PATHS in ACTION 1
PATHS in ACTION 2
PATHS in ACTION 3
Focus on Implementation Science
The way a programme is implemented
is more important than the programme
itself
• Implementation is an emerging Science- Journal
•
•
of Implementation Science appeared in 2006
Barbara Kelly and Danny Perkins
(2012),Cambridge Handbook of Implementation
Science for Psychology in Education, Cambridge
University Press
Offers conceptual and applied frameworks and
strategies for effective implementation
13
Basic Implementation Strategies (Blasé
et al 2012) Who, why and what in
building effectiveness
1.
2.
3.
4.
5.
6.
7.
Recruitment and selection: choose those who are interested
in the programme and keen to be involved.
Training: provide in depth pre and in service training on both
theory and practice issues linked to the programme
On going Consultation and Coaching: follow through initial
training with coaching support in context for practitioners.
Monitor and evaluate Staff performance: monitor how staff
use skills to target input at all stages
Inform decision making and policy review: provide data to
support on-going local investment decisions
Facilitative administrative support: have clear leadership and
support to implement effectively and sustain programmes
effectively
System Support: Ensure the availability of financial,
organisational and other resources- don’t run on a shoe string
14
Design of the project
Every new implementation of any programme presents a
new ecology and different strengths and weaknesses
• Two year implementation process
• This reports on first year audit
• Used Audit Methodology - support and check the
•
•
progress made in implementing evidence based
programmes and allows feedback and changes
to improve effectiveness
Monitors successes and obstacles along the way
Poor implementation results in low impact
15
So what have we actually done?
• Following the Implementation Drivers identifies by Blase et al 2012
we focused on key areas to support implementation:
First -Set up a steering group involving Principal Psychologist, Senior
EP, EPs, Quality Improvement Officer, Early Years Co-ordinator to
guide the implementation processes
Recruitment and initial training processes for staff
• Presentation to volunteer nursery head teachers
• Completion & return of the Readiness Checklist
• Selection of nurseries
• 4 nurseries with 10 children in each as the in-depth research group
16
Assessing Readiness Aarons et
al 2012
• Readiness for evidence based practice is a
•
•
concept developed by Aarons
Arose from medical context where practitioners
were unable or unwilling to implement evidence
based programmes successfully
Reasons are linked to processes outlined earlier:
values , attitudes, understanding, resources,
leadership………and `diamond hard resistance`
17
PATHS readiness checklist
• Asked volunteers to consider a range of
issues which they would have to address
to implement PATHS effectively
• This was used to select candidates for the
study
• Not an infallible method as reality always
proves more challenging but alerts
practitioners to implementation demands
18
Extensive data collection
throughout and at end of year one
Impact on children
•
•
•
•
•
•
Kusche Emotional Inventory (Domitrovich et al)
The British Picture Vocabulary Scale 3
Demographic Questionnaire
Preschool Behaviour Checklist
Number of children involved was based on the statistical
analysis of the Domitrovich study – 18 children required
to demonstrate effects.
Within Subjects Design to evaluate the impact of Preschool PATHS This design makes the assumption that we
know Pre-school PATHS works and looks at its effects on
individual children in different early years contexts.
19
Implementation, training and
support activities
• Major innovation in Glasgow South -Coaching and
•
•
•
•
•
Modelling by Peripatetic Preschool Team
Inset to all nursery staff on Pre-school PATHS &
Implementation Issues
Signed agreement to follow implementation guidelines
Discussion with staff about frequency and quality of
lessons (30 required for significance)
Observation of some lessons for fidelity and engagement
lessons
Interview with staff about impact after year one
20
Statistical Analysis after one
year
• Repeated measures ANCOVA looked at scores in Kushe
•
•
•
Emotional Inventory and the British Picture Vocabulary
Scale (BPVS).
The co-variants controlled were: gender, age, number of
lessons taught, English as an Additional Language (EAL),
ethnicity and Additional Support Needs (ASN).
Firstly the nursery results were looked at together as a
whole population (n=30).
Only thirty of the 40 target children had a full data set
21
Emerging positive trends
• The results of the ANCOVA for the Kushe
Emotional Inventory showed no
significant overall effect of time at this
stage; F(1,23)=0.881, p=0.358.
• However, the mean percentage of correct
answers on both did increase from time 1
(mean= 59.83%) to time 2
(mean=63.33%).
22
Checking by establishment
• T- Test significant on both the BPVS and
Kushe in two establishments but low
numbers of children hampers confident
interpretation
• Backward trends began to emerge in two
establishments
23
Interim and End of Year One Outcomes
Staff views via telephone interviews
• Overall feedback is very positive.
• All note positive changes in children's emotional and
social development and behaviour.
• All felt PATHS achieved its theoretical objectives very
well.
• Respondents emphasised key processes of children`s
identification and empathy with puppet characters and
the value of scenarios played out by characters which
generalised to real situations with teacher prompts
• All will continue into year two and report developing
confidence in delivery and management of the lessons
24
Little data to date on the early stages of implementationthe installation and initial implementation phases (Schaffer,
Nesselrodt and Stringfield (1997)
• Many programmes fail during the installation
•
phase because of lack of investment of time and
resources
Leadership role is crucial in maintaining a focus
on quality implementation and challenging
current values, beliefs and paradigms which do
not fit the new way of working
• No one has a crystal ball and this phase is characterised
by rapid problem solving cycles which respond to issues
emerging in the audit process
25
Why were results not reaching significance?
Audit of early implementation stages
Training-not all received follow up training
Number of lessons- no establishment
carried out the planned thirty lessons
Coaching- varied in style, frequency and
experience: some staff began doing lessons
themselves when the agreement had been
on modelling in the early stages, some did
not find the coach helpful.
26
Implementation and child
results
• These are reassuringly in a strong positive
direction in two establishments
• In the other two establishments problems
arose probably impacting on child results
Staff continuity, knowledge and experience
of coaches, knowledge and training of staff,
frequency of lessons, resource issues and
confidence, fidelity of lessons taught.
27
Staff reactions to Implementation
Issues
• Majority nurseries felt that the in-service presentation
•
•
•
•
•
•
and taster sessions fired imagination
Coaches too slow to get off the ground re lessons
Some difficulties with language barriers – high numbers
of EAL parents and children in one nursery
Two nurseries reported feeling unsupported initially and
that the coach had poor skills
Resource and time issues re organisation and
photocopying
Difficulties across job share staff and communication
with them
Extension activities were too difficult.
28
Reaction of parents
• Nurseries reported that parents seemed to be
•
•
•
interested and enthusiastic but they hadn’t set
out to involve them
One set up a PATHS display and this had
stimulated a lot of conversation
One reported that parents had come and asked
to see the materials
Staff feel that parent involvement would be
helpful in the implementation phases
29
What’s gone well?
• Most reported coach had been excellent and
•
•
•
•
built confidence in staff to deliver on their own
Programme is really clear and very
comprehensive
Materials are excellent and children are really
enjoying the programme
Two nurseries report evidence of children
discussing PATHS at home with parents
Three feel impact is clear already
30
Next phase
• Each establishment to complete 30 lessons before child assessment
• Iron out problems via close collaboration between nursery EP and
staff
• Carry out a final audit after two years or thirty lessons and when
staff are seen to deliver lessons with fidelity and confidence
• Established PATHS Network to foster good implementation of
PATHS and other programmes, promote self evaluation and to
develop links to global PATHS networks
31
Implementation Science: The
Practitioner is the intervention
• Offers a sound evidence base and guidelines for building
•
•
accountable evidence based practice across all aspects
and levels of Psychological Service Delivery
Success for children ultimately depends on staff
readiness to deliver high quality evidence based
teaching, care and programmes with enthusiasm, depth
of understanding of theory and skill.
Supporting those processes is a major part of our work
but is demanding and intricate.
32