Transla'ng research into prac'ce for socially disadvantaged families: Lessons from a five-­‐year evalua'on of a peer-­‐led paren'ng programme Dr Crispin Day & Dr Daniel Michelson CAMHS Research Unit, Ins:tute of Psychiatry, King's College London; Centre for Parent and Child Support, SLaM NHS Founda:on Trust CAMHS EBPU Seminar Anna Freud Centre, UCL June 20th 2013 The Good News ...But n  Families oQen do not get the Parent training is an effec:ve interven:on (NICE, 2013) n
n
n
n
help they need, par:cularly vulnerable and minority families Scale of need far outweighs the service capacity available Complex referral pathways and s:gma Service disinvestment Effec:ve under rou:ne condi:ons? Does PT work in the “real world”? n  Meta-­‐analysis: Michelson et al. (2013), Clinical Child and Family Psychology Review n  Effects of PT vs WLC on child disrup:ve behaviour tested across four real-­‐world condi:ons (opera:onal criteria based on Weisz et al., 1995) n  Clinic-­‐Referred Sample n  Rou:ne Secng n  Rou:ne Service n  Non-­‐specialist Therapists n  N=28 eligible RCTs classified according to presence/absence of each criterion and total no. of criteria present Meta-­‐analysis: Results n  Only 3 RCTs scored 4/4 (in 22 years of research!) n  Even studies iden:fied in :tles as “effec:veness trial,” n
n
n
n
n
“pragma:c” or “prac:ce-­‐based” did not fulfil all criteria Majority of samples were self-­‐referred, suburban, Caucasian and middle-­‐class Evidence of publica:on bias (small studies with large effects were rela:vely over-­‐represented) Sig. overall advantage for PT Sig. between-­‐study heterogeneity; not accounted for by total real-­‐world criteria scores in meta-­‐regression analysis (p = 0.98 for ECBI-­‐I) No consistent differences in effec:veness for sub-­‐analyses grouped by individual criteria Results: Effects of PT on ECBI-­‐I (N=24 trials) n  Random effects meta-­‐analysis n  Heterogeneity chi-­‐
squared = 79.55 (d.f. = 23) p < 0.001 n  I-­‐squared (varia:on in WMD aoributable between-­‐study as opposed to within-­‐
study differences) = 71.1% Study
%
ID
WMD (95% CI)
Weight
Axberg 2012
-18.50 (-33.13, -3.87)
4.22
Connell 1997
-41.67 (-55.99, -27.35)
4.27
Day 2012
-6.67 (-16.79, 3.45)
5.04
Enebrink 2012
-21.93 (-35.74, -8.12)
4.37
Feinfeld 2004
-12.30 (-29.30, 4.70)
3.80
Forehand 2011
-12.62 (-27.05, 1.81)
4.26
Gallart 2005
-20.90 (-45.61, 3.81)
2.67
Gardner 2006
-17.80 (-34.50, -1.10)
3.85
Gross 1995
8.20 (-9.53, 25.93)
3.68
Hutchings 2007
-21.70 (-33.14, -10.26)
4.80
Joachim 2010
-20.76 (-37.94, -3.58)
3.77
Kling 2010
-20.90 (-32.02, -9.78)
4.86
Larsson 2009
-20.80 (-34.01, -7.59)
4.48
Leung 2003
-29.17 (-43.01, -15.33)
4.36
Markie-Dadds 2006
-46.23 (-59.14, -33.32)
4.53
Morawska 2011
-27.56 (-40.82, -14.30)
4.47
Nixon 2003
-23.11 (-36.83, -9.39)
4.39
Patterson 2002
-4.60 (-13.69, 4.49)
5.23
Sanders 2000
-28.41 (-37.49, -19.33)
5.23
Schumann 1998
-52.10 (-74.19, -30.01)
3.01
Taylor 1998
-17.20 (-34.02, -0.38)
3.83
Turner 2006
1.83 (-16.11, 19.77)
3.64
Webster-Stratton 1990
-10.65 (-31.22, 9.92)
3.23
Webster-Stratton 1997
-36.84 (-52.62, -21.06)
4.01
Overall (I-squared = 71.1%, p = 0.000)
-20.90 (-26.26, -15.53)
100.00
NOTE: Weights are from random effects analysis
-50
-25
0
25
50
Meta-­‐analysis: Conclusions §  Child disrup:ve behaviour problems can respond effec:vely §
§
§
§
to PT, even when delivered under broadly defined real-­‐
world prac:ce condi:ons. BUT there may s:ll be more differen:ated criteria under which PT is +/-­‐effec:ve More consistent and detailed repor:ng on care provider/
par:cipant characteris:cs is needed to inform decisions about whether an EBI is effec:ve in general (for one or more defined service user groups) and/or in par7cular (for an individual service user) More evidence is needed on: (1) effects in diverse pop’s; (2) effects under genuine prac:ce condi:ons (outside of research trials); (3) characteris:cs of non-­‐responders; (4) implementa:on outcomes And so… Empowering Parents, Empowering Communi:es (EPEC) EPEC is an innova:ve, low cost method of working with socially disadvantaged communi:es to ensure that they get effec:ve, evidence based early paren:ng interven:on It is a collabora:on between South London and Maudsley NHS Founda:on Trust, King’s College London and our local communi:es in South London What is EPEC? n  We recruit, train and support local parents in South London to run paren:ng groups in their communi:es n  The “peer facilitators” learn to use our manualised programme called Being A Parent n  EPEC peer facilitator training consists of 60 hrs over 10 weekly workshops n  Training is accredited by the Open College Network, equivalent to 18 NVQ credits n  AQer training, peer facilitators con:nue to receive supervision and further training from EPEC EPEC programme n  Delivered using a structured manual n  based on aoachment, social learning, rela:onal, and cogni:ve-­‐behavioural principles and methods n  Sessions involve n  sharing of informa:on, group discussion, demonstra:on, role play, reflec:on and planning/
review of homework tasks EPEC programme n
n
n
n
n
n
n
n
Session 1: being a parent n  “Good enough” versus “perfect” parent n  Taking care of ourselves Session 2: feelings, communica7on, and culture n  Remembering what it was like to be a child n  Acknowledging, accep:ng, and expressing feelings Session 3: play and listening n  Non-­‐direc:ve play (“special :me”) n  Prac:sing listening Session 4: labels and praise n  Avoiding “labels” when describing behaviour n  Using descrip:ve praise to change behaviour Session 5: understanding children’s behaviour n  Understanding children’s behaviour in response to needs n  Discipline Session 6: seHng boundaries n  Understanding boundaries n  Rewards n  Asser:ve versus aggressive behaviour n  Time out, challenging, and saying no Session 7: listening Session 8: review and coping with stress So far in the UK we have... Trained Over 30 peer facilitators Wide range of ethnic and cultural backgrounds Facilitators have translated Being A Parent into African French and Somali So far in the UK we have... They have gone on to run over 65 courses for over 800 parents reaching nearly 2,000 children EPEC Being A Parent courses Are designed to: §  Improve parent-­‐ child rela:onships §  Reduce child behaviour problems §  Increase paren:ng confidence EPEC Being A Parent courses Each course runs for eight 2 hour sessions with between 7-­‐14 parents in a group EPEC Being A Parent courses Are run in primary schools, children’s centres, community and faith centres for parents who are having difficul:es with a child aged 2-­‐11 years But does EPEC work? §  Ongoing programme of research and service evalua:on §  Data on 873 par:cipants (although quality is variable) §  Four linked studies: §  Pilot evalua:on of peer facilitator training and early groups in Southwark (Day, Michelson et al., 2012a, Child and Adolescent Mental Health) §  RCT in Southwark (Day, Michelson et al., 2012b, BMJ) §  Evalua:on/benchmarking of rou:ne outcomes and predictors of effec:veness (Charalambides, 2013, D.Clin.Psy. Project) §  Qual study of peer facilitators’ experiences (Thomson et al.) §  In prepara:on: forma:ve evalua:on of EPEC for parents of adolescents §  In progress: grounded theory study of social capital and its development within EPEC groups; more secondary analysis of quant data Looking at the bigger picture: Evalua:on framework and ques:ons Taxonomy of outcomes in implementa:on research (Proctor et al., 2011) Service outcomes: To what extent is it effec:ve (in controlled research and in prac:ce)? Equitable (in terms of uptake and outcomes)? Implementa:on outcomes: To what extent is it acceptable? Appropriate? Sustainable? Adopted in different secngs? And what does it cost? Effec:veness of EPEC system: What are the effects of peer facilitator training on learning outcomes? Pilot study (2012a): 31 peer facilitators (30 females; aged 27-­‐49 years, M=37.4 years) enrolled in two training cohorts (2009-­‐2010) §  22 went on to complete accredita:on and run at least one group §  Significant pre-­‐post increases in knowledge of posi:ve paren:ng/group facilita:on principles (23-­‐item mul:ple choice measure) and self-­‐efficacy (21-­‐item measure) §
Effec:veness of EPEC: What are the effects of the interven:on on parent and child outcomes? Pilot study (2012a): §
Sig. pre-­‐post improvements on SDQ Total Difficul:es scale (N=32), SDQ Conduct Problems subscale (N=34) & Concerns About My Child (N=30) RCT (2012b): EPEC vs WLC §
§
§
§
§
N=116 (59 EPEC, 57 WLC) Six sites (three schools, two children’s centres/schools, and one church) across Southwark during 2010 Inclusion criteria: Parental caregivers seeking help with managing the problem behaviours an index child aged 2-­‐11 years in the absence of neurodevelopmental probs Sig. advantage (ITT analysis) for EPEC on ECBI-­‐I (d=0.38), ECBI-­‐P (d=0.56), Concerns About My Child (d=0.77), SDQ Total (d=0.29), Paren:ng Scale (d=0.69) Non-­‐sig. between-­‐group difference on Paren:ng Stress Index RCT: Selected results 1 3.5 3.34 Parents’ paren:ng improves 3.26 3.23 Interven'on 3 2.91 (Paren:ng Scale: Arnold & O’Leary) 2.5 Before Effect size=0.69 AEer Control RCT: Selected results 2 60 53.56 Parents achieve their paren:ng goals (Concerns About My Child: Scoo et al., 2001) 50 52.38 50.48 40 30 25.72 20 Interven'on Control 10 Before Effect size=0.77 AEer Effec:veness of EPEC: How effec:ve is the interven:on in rou:ne service delivery? Rou:ne service evalua:on (2013): §
§
§
§
§
Based on 8 groups (N=109) delivered in Southwark, Greenwich and Lambeth during autumn term of 2011 Paired CAMC scores available for N=60 Large, significant pre-­‐post improvement Effect size for CAMC (1.14; CI = 0.62 -­‐ 1.66) Cf. RCT (uncontrolled ES = 0.88; CI = 0.57 -­‐ 1.20). Equity: What is the uptake among BME and socially disadvantaged groups? RCT (N=116): §  71% from BME groups §
§
cf. 35% BME residents in Southwark’s general popula:on BME families under-­‐represented in Southwark CAMHS §  67% in lowest 20% of UK pop. based on disposable household income §  13% owner occupiers Rou:ne service evalua:on (N=109): §  75% from BME groups (ranged from 73-­‐85% across 3 boroughs) §  73% unwaged (range 63-­‐76%) §  41% lone parents (range 38-­‐41%) §  49% English as second language (range 41-­‐52%) Equity: What family characteris:cs are associated with outcomes? n  Main effect of ethnicity: F n
n
n
n
(1,51) = 65.369, p <0.001 Sig. differences between ethnic groups on Time 2 scores No sig. interac:on between ethnicity and :me Pairwise comparisons showed no sig. differences in mean change score post-­‐interven:on between ethnic groups No main effect/interac:on for any other family characteris:c Acceptability: To what extent are peer facilitators and par:cipants sa:sfied with EPEC? Pilot study: §  100% of peer facilitators “a great deal” / “quite a lot” sa:sfied with training overall (although they asked for more :me on skills prac:ce) RCT: §  100% of par:cipants were “a great deal” / “quite a lot” sa:sfied overall §  Median acceptability score of 4 (max possible) for every item on the Acceptability Ra:ng Scale §  Perceived competence of facilitators §  Ability of facilitators to relate to the group §  Mo:va:onal quali:es of facilitators §  Increased understanding of posi:ve paren:ng, development of posi:ve paren:ng skills, confidence in the paren:ng role Appropriateness: To what extent is EPEC relevant/
compa:ble/prac:cable for par:cipants? Forthcoming study: §  Will obtain in-­‐depth data on par:cipants’ experience of EPEC groups with a focus on dimensions of social capital But indirectly: “They voted with their feet” (Lenin, 1918) §  Reten:on rate of 92% in RCT (M=7.1 sessions aoended, SD=0.9) §  cf. 75-­‐85% in two other UK RCTs of professional-­‐led PT in socially disadvantaged areas (Hutchings et al., 2007; Scoo et al., 2001) §  Comple:on rate of 73% in rou:ne evalua:on (M=5.4 sessions aoended, SD=2.5) §  Achieved despite less stringent methods to gauge appropriateness at outset, and fewer resources to maintain involvement Sustainability: What factors influence successful prac:ce by peer facilitators and their sustained par:cipa:on in EPEC? Qualita:ve study (Thomson et al., in prep.): interviews with N=14 peer facilitators Three themes: n  Knowledge and Skills Development n  Personal Transforma:on n  Facilitator Quali:es Knowledge and Skills Development (1) Supervision and reflec:on enable transla:on of knowledge/skills into prac:ce §  Logis:cs also important: Par:cipants emphasised the importance of admin support, reliable crèche workers, a comfortable venue free from disrup:ons and coopera:on from other staff members at the venue §  Child safeguarding was a par:cularly challenging area of prac:ce, even among most experienced facilitators §  EPEC manual was seen as a valuable resource that strengthened their credibility with the aoendees because: §
“the parents need to have a sense that I am referring to something, and it’s not just all my own ideas” Knowledge and Skills Development (2) §
Both formal and informal supports were significant in building peer facilitators’ confidence, effec:veness and sense of belonging in EPEC: You've got [EPEC supervisors] who you can call up at any 7me and ask anything, or who are going to call you and ask you stuff… then you've got all the people I trained with, it’s really nice to be able to compare with them …I think everyone collec7vely is important in terms of support §
The system of pairing new and experienced co-­‐facilitators was generally reassuring and enabled modelling of good prac:ce: I valued anything [my co-­‐facilitator] said to me … she let me be me, she didn’t try and impose her ways on me at all … but at the same 7me modelled a really good example Personal Transforma:on n  Peer facilitators described deep personal effects of par:cipa:ng in EPEC n  Processes of comple:ng accredita:on and working as a peer facilitator were associated with improved self-­‐belief, self-­‐worth, confidence and the possibility of new opportuni:es It’s given me confidence and a different way of life – not to be a housewife all my life n  Experience of facilita:ng changes in other parents contributed to a sense of achievement and emo:onal reward n  Development of new knowledge, skills and confidence also influenced peer facilitators to seek opportuni:es outside of EPEC, such as going on to train as a teaching and nursery assistant I enjoy seeing them [parents] bloom and change, I enjoy talking to the parents, learning from them Facilitator Quali:es (1) §
Strong personal iden:fica:on and enthusiasm about EPEC’s ethos and methods EPEC just became a very important piece of me I want them, parents, to understand the meaning of being a parent, and how to bring them up, because children are very vulnerable, they are like soX dough, and if you shape it properly, you will have a very strong, confident, disciplined person, who will go out in the world, and behave the way they have been taught by their parents, and make a beYer community Facilitator Quali:es (2) §
Personal quali:es of genuineness and modesty enabled helping rela:onships that did not rely on adop:ng an “expert” stance We’re all in this together, I’m not coming in as an expert, I’ve picked stuff up along the way that I want to share with you but I want to learn from you too §
Peer facilitators believed strongly that EPEC was more acceptable to local parents because it was peer and not professionally led The person that’s facilita7ng it actually experiences the same thing as the person he’s trying to send the message across to, and that makes it helpful to explain Implica:ons of qualita:ve study §
High-­‐quality peer-­‐led services require careful selec:on of peer leaders who are sufficiently enthusias:c and reflec:ve in order to overcome inevitable personal and logis:cal challenges & make best use of available support systems §
EPEC has been generally effec:ve in maintaining a core group of skilled peer facilitators who iden:fy strongly with the programme and are passionate about its poten:al to transform the lives of parents and families §
This tes:fies to careful selec:on procedures and quality of training and supervision §
Increasing dissemina:on of EPEC and other peer-­‐led approaches requires systema:c monitoring of these aspects of service delivery alongside more commonly measured “pa:ent” outcomes Adop:on: To what extent has EPEC been implemented in different secngs/contexts? To date: §  Southwark: 51 groups §  Lambeth: 3 groups §  Greenwich: 5 groups §  Croydon: 8 groups Other developments: §  “Living with Teenagers”: groups delivered by (and for) parents of adolescents §  “Baby EPEC” What does it cost? §  Cost-­‐effec:veness studies are s:ll needed But: §  Direct costs of running the programme (including facilitators’ wages, travel expenses, prin:ng of materials, accredita:on, crèche workers, refreshments and rental of a room) were about £2,700 per group §  cf. Conven:onal parent training costs es:mated at £1,200 per par:cipant; can produce savings to society of £16,435 per family over a 25-­‐year period (Bonin et al., 2011) Conclusions n  Example of mental health service, academic and community n
n
n
n
n
n
partnership Feasible, popular, (cost-­‐)effec:ve peer-­‐led interven:on Good performance in replica:on sites and rou:ne prac:ce Systema:c evalua:on (incl. trials, qual studies and rou:ne monitoring) has been cri:cal to its dissemina:on and adop:on Measuring “success” of implementa:on requires aoen:on to mul:ple outcomes and varied methods Now addressing issues of spread and replica:on Developing new versions Acknowledgments §  EPEC programme leads: Caroline Penney & Lucy §
§
§
§
§
§
Draper Outreach workers: Caroline Adewole & Cathy Kearney All the facilitators and other EPEC staff Postdoctoral research workers: Stacey Thomson, Alana James, Anke Franz Research assistant: Ilan Ben-­‐Zion Doctoral students: Monica Charalambides, Sophie Hall Funders: Guy’s & St Thomas’ Charity, Maudsley Charity